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SUBJECT ; 


ATTACHMENT : 


Publication  -  Integrated  Quality  Control  Sample 
Designs:   National  Conference  Proceedings 

The  attached  publication  provides  a  comprehensive 
overview  of  the  National  Conference  on  Integrated 
Quality  Control  Sample  Designs  which  was  held  in 
Washington,  D.C.  on  February  1^-16,  1978.   The 
Conference  was  jointly  sponsored  by  the  Department 
of  Agriculture  and  the  Department  of  Health,  Education, 
and  Welfare.   The  expressed  purpose  of  the  conference 
was  to  provide  a  forum  for  the  interchange  of  ideas 
and  experiences  concerning  integrated  QC  sample 
designs. 


The  increased  emphasis  on  monitoring  efforts  in 
recent  years  has  lead  to  expanded  sample  size  and 
universe  requirements  in  the  Food  Stamp  and  Medicaid 
Quality  Control  (QC)  Systems.   As  a  result,  overlap 
in  the  populations  subject  to  QC  sampling  has  occurred. 
Thus  a  single  family  may  be  subject  to  selection  and 
review  under  each  of  the  three  QC  systems.   Concern 
over  the  cost  of  such  systems  as  well  as  the  un- 
necessary imposition  on  the  recipient  population 
has  lead  to  increased  interest  in  the  potential 
that  an  integrated  QC  sample  design  may  hold. 
Through  integrated  sampling,  a  family  which  partici- 
pates in  multiple  programs  is  selected  and  reviewed 
simultaneously  for  each  program  requiring  a  Quality 
Control  review. 

States  which  have  instituted  integrated  sample  designs 
were  invited  to  make  selected  presentations  on  their 
experiences,  i.e.,  cost  effectiveness,  impact  upon 
the  review  process,  and  sample  designs.   In  addition. 


Social  Security  Administration 

Office  of  Management  and  Administration 

Office  of  Quality  Assurance 


technical  experts  representing  the  various  programs 
were  asked  to  make  presentations  on  specific  aspects 
of  integration.   Planning  efforts,  recent  developments 
in  the  various  QC  systems,  sample  design  methods,  and 
the  criteria  for  submission  of  integrated  sample  design 
proposals  were  discussed  in  detail.   Workshops  on 
sampling  techniques  and  the  case  review  process  were 
also  held  in  order  to  provide  participants  with  the 
opportunity  for  open  discussion. 

It  is  anticipated  that  States  will  become  increasingly 
interested  in  sample  integration  as  a  mechanism  to 
effect  cost  savings.   Although  potential  savings  do 
exist,  there  are  statistical,  policy,  and  procedural 
issues  which  must  be  resolved.   Each  State,  however, 
is  in  the  best  position  to  make  its  own  decision 
regarding  the  feasibility  and  potential  benefits  of 
such  an  approach  based  on  its  own  particular  circum- 
stances.  We  trust  that  the  conference  and  the  attached 
document  will  assist  States  in  making  this  decision. 

Additional  copies  of  the  conference  proceedings  may 
be  obtained  by  contacting  the  Division  of  Quality 
Control  Management,  330  C  Street,  S.W.,  Room  1416, 
Mary  E.  Switzer  Building,  Washington,  D.C.   20201. 


INQUIRIES  TO: 


SSA  Regional  Commissioners 


'ommissioner 
for  Office  of  Family  Assistance 


Section  I  -  Conference  Overview 

On  February  1-4-16,  1978,  the  DHEW  and  the  Department 
of  Agriculture  jointly  sponsored  a  national  conference 
on  integrated  Quality  Control  sample  designs.   In  view 
of  expanding  sample  requirements  upon  States  and  the 
potential  savings  associated  with  integrated  sample 
designs,  approximately  150  Federal  and  State  employees 
were  in  attendance. 

The  expressed  purpose  of  the  conference  was  to  provide 
a  forum  for  the  interchange  of  ideas  and  experiences 
concerning  integrated  QC  sample  designs.   In  the  interest 
of  furthering  this  purpose,  a  comprehensive  overview  of 
the  conference  is  being  distributed  to  all  interested 
parties.   It  should  be  noted,  however,  that  some  of  the 
materials  may  no  longer  be  current  and  are  being  presented 
for  historical  reference  only. 

The  Departments  wish  to  express  their  appreciation  to  the 
presenters  and  participants  for  theirJ contribution  to  the 
success  of  the  conference. 
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Welcoming  Remarks  and  Conference  Format 

Conference  participants  were  welcomed  to  Washington,  D.C.  by  the 
following  officials  from  the  sponsoring  agencies. 

Barry  Van  Lare ,  Associate  Commissioner 
Office  of  Family  Assistance 
Social  Security  Administration 
Coordinator  of  Welfare  Reform  for  DHEW 

Nancy  Snyder,  Acting  Deputy  Administrator 
Family  Nutrition  Program 
Food  and  Nutrition  Service 
Department  of  Agriculture 

Tom  Staples,  Acting  Director 
Family  Assistant  Studies  Staff 
Office  of  Research  and  Statistics 
Social  Security  Administration 

Victor  Kugajevsky,  Acting  Deputy  Director 

Office  of  Financial  Management 

Medicaid  Bureau 

Health  Care  Financing  Administration 

John  X.  Bowes,  Director 

Division  of  AFDC  Quality  Control  Management 

Office  of  Quality  Assurance 

Social  Security  Administration 

The  welcoming  officials  emphasized  the  need  for  a  continuing  focus  on 
program  integrity  and  expressed  their  support  for  the  conference 
objectives.   It  was  particularly  noted  that  potential  savings  can 
result:  from  integrated  Quality  Control  sample  designs  and  that  it 
offers  Federal  and  State  staff  the  unique  opportunity  to  work  closely 
together  in  realizing  these  savings. 

Welcoming  remarks  were  followed  by  a  series  of  presentations  designed 
to  provide  an  overview  of  specific  aspects  of  sample  integration.  In 
addition,  conference  participants  had  the  opportunity  to  break  up  into 

specialized  workshops  so  that  issues  of  concern  could  be  discussed  in 
greater  detail. 


Management  Summary 

Rationale  for  Integration 

States  are  required  to  perform  Quality  Control  (QC)  reviews  in  three 
State  operated  programs;  Aid  to  Families  with  Dependent  Children  (AFDC), 
Medicaid,  and  Food  Stamp  (FS).   While  these  programs  are  separate,  they 
are  quite  similar  in  terms  of  eligibility  standards  and  QC  review 
methodology.   The  same  family  unit  may  participate  in  two  or  more  programs 
at  the  same  time.   As  of  July  1,  1976,  the  Department  of  Agriculture 
increased  its  sampling  requirements  to  include  AFDC  as  well  as  non-public 
assistance  households.   Also,  the  Health  Care  Financing  Administration  has 
recently  expanded  its  sample  requirements  to  include  AFDC  recipients.   As 
a  result,  overlaps  have  been  created  in  the  sample  universes  of  these  three 
programs  and  the  same  family  unit  may  be  subject  to  selection  and  review 
under  each  QC  system.   Concerns  about  the  costs  of  such  systems  and  the 
overlap  of  populations  surveyed  have  led  to  increased  State  interest  in 
the  potential  savings  that  an  integrated  sample  design  and  review  process 
may  hold. 

Integrated  sampling  is  a  technique  by  which  a  sample  of  family  units  (or 
cases)  which  participate  in  multiple  programs  is  selected  and  reviewed 
simultaneously  for  each  program  requiring  a  Quality  Control  review. 
Ideally,  the  largest  possible  statistically  valid  sample  of  multi- 
participation  cases  is  selected  in  order  to  maximize  the  use  of  available 
information.   The  total  number  of  QC  review  cases,  home  visits,  and 
collateral  contacts  is  reduced;  resulting  in  a  potential  savings  to  the 
State  in  time,  travel  costs,  and  program  efficiency.   However,  there  are 
many  statistical,  policy,  and  procedural  issues  associated  with  sample 
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integration  and  each  State  must  make  its  own  decision  regarding  the 
possible  benefits  of  integration  based  on  its  particular  situation.   To 
date,  eleven  States  have  approval  for  integrated  sample  designs  and  many 
more  States  have  expressed  interest  in  this  approach. 

Potential  Cost  Savings 

Quality  Control  sample  integration  does  offer  the  potential  for 
significant  cost  savings  to  the  States.   Montana,  for  example,  realized 
a  savings  of  15,000  travel  miles  in  the  first  six  months  of  its  integration 
of  AFDC  and  Food  Stamp  QC  samples.   At  a  rate  of  55  miles  per  hour,  this 
represented  a  savings  in  staff  time  of  38  working  days  over  the  six  month 
period.   Actual  dollar  savings  in  mileage,  meals,  and  lodging  amounted  to 
$3,350;  a  29%  cost  reduction  over  the  previous  period. 

In  addition  to  savings  in  travel  costs,  sample  integration  results  in  a 
reduction  in  the  number  of  cases  which  must  be  reviewed.   Integration  of 
the  three  QC  samples  ( AFDC/FS/Medicaid)  currently  being  tested  in  West 
Virginia  results  in  a  49%  reduction  in  the  number  of  required  case  reviews. 
Although  an  integrated  case  review  requires  more  time  to  complete,  it 
requires  less  time  than  if  the  two  or  more  integrated  reviews  were  completed 
separately.   Illinois  estimated  that  an  integrated  AFDC  and  Food  Stamp 
■review  required  24%  less  time  to  complete  than  if  the  two  reviews  were 
completed  independently.   In  Rhode  Island,  reviewer  productivity  increased 
from  9.9  reviews  to  15.2  reviews  per  month  per  reviewer  in  the  second  six 
month  period  of  its  AFDC/FS  sample  integration.   Additional  cost  savings 
can  be  realized  by  stratifying  the  sample  in  order  to  increase  the  proba- 
bility of  selecting  cases  with  a  lower  cost  per  review  ratio. 
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Many  variables  and  particularly  the  State's  own  situation  effect  potential 
savings.   A  small  or  predominately  urban  State  may  not  achieve  the  savings 
in  travel  costs  realized  in  a  geographically  large  State  such  as  Montana. 
On  the  other  hand,  a  large  urban  State  may  realize  a  greater  increase  in 
the  number  of  reviews  completed  per  reviewer.   Some  States,  although  not 
integrating  their  QC  samples,  have  realized  a  cost  savings  simply  through 
the  use  of  generic  as  opposed  to  specialized  QC  reviewers. 

Impact  on  the  Review  Process 

Although  potential  cost  savings  exist,  there  are  many  policy  and  procedural 
issues  which  must  be  taken  into  account  when  considering  sample  integration. 
There  may  be  implications  for  staffing  and  training  in  that  reviewers  must  become 
proficient  in  two  or  more  eligibility  programs  as  well  as  the  corresponding  QC 
review  policies  and  procedures.   This  may  present  special  problems  to  States 
that  have  separate  program  manuals  or  complex  State  policies.   In  addition, 
reviewers  currently  must  complete  a  separate  and  detailed  worksheet  for  each 
QC  program  for  which  a  case  is  being  reviewed  even  though  much  of  the 
recorded  information  is  the  same  or  similar.   States  consider  this  dupli- 
cation the  single  most  difficult  and  time  consuming  aspect  of  the  integrated 
review  process.   While  several  States  are  experimenting  with  an  integrated 
worksheet  (one  which  can  be  used  for  all  QC  programs  under  review) ,  none 
have  been  successfully  field  tested  to  date. 

The  fact  that  the  various  QC  programs  are  administered  separately  at  the 
Federal  level  is  also  a  complicating  factor.   States  must  maintain  the 
capacity  to  reach  independent  review  findings  for  each  of  the  integrated 
QC  programs  so  that  the  different  program,  case  completion,  and  reporting 
requirements  can  be  met.   Changes  in  one  QC  program  may  have  a  detrimental 
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effect  upon  sample  integration  if  not  closely  coordinated  at  the  Federal 
level.   Recent  changes  in  Food  Stamp  legislation  will  dramatically  affect 
the  distribution  of  program  benefits  and  may  require  subsequent  changes 
in  their  FS-QC  program.   The  recent  shift  in  the  AFDC-QC  review  cycle  is 
also  likely  to  cause  complications  for  sample  integration  should  AFDC  and 
Food  Stamp  sampling  periods  remain  on  different  review  cycles. 

However,  States  experienced  in  sample  integration  believe  that  it  does 

offer  practical  advantages  to  the  review  process.   States  maintain  that 

in  addition  to  increased  productivity,  sample  integration  increase 
reviewer's  interest  and  upgrades  the  job  function. 

Integrated  Sample  Designs 

There  are  essentially  five  sample  designs  available  whereby  cases  selected 
for  one  QC  sample  can  also  be  utilized  as  part  of  one  or  two  other  QC 
samples.   Each  design  has  its  own  particular  requirements,  relative 
efficiency,  advantages,  and  disadvantages.   Several  of  the  methods  have 
stratified  sample  frames  and/or  weighting  requirements.   The  alternative 
methods  may  range  in  relative  efficiency,  i.e.,  the  percentage  of  reduction 
in  the  number  of  required  case  reviews,  from  24%  to  -49%.   One  method 
randomly  selects  cases  by  social  security  number  and  produces  self-weighting 
samples  without  requiring  that  the  program  benefits  of  the  cases  in  the 
sample  universe  be  known.   Another  method,  the  Sequential  Random  Sample 
Method,  can  result  in  a  49%  reduction  in  the  number  of  required  case 
reviews  and  does  not  require  stratifying  sample  frames  prior  to  sampling. 
It  does,  however,  require  that  weighting  be  used  in  the  compilation  of  QC 
findings.   An  Optimum  Allocation  Method  is  designed  to  minimize  the 
variance  of  error  rate  estimates  and  the  proportional  costs  of  sample 
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cases.   It  has  stratified  sample  frames  and  weighting  requirements.   As 
sample  methodologies  become  more  complex,  however,  computer  capability- 
may  become  an  important  consideration. 

A  major  sampling  issue  is  the  objective   of  sampling  requirements. 
Current  fixed  sample  size  requirements  are  being  questioned  in  favor  of 
sampling  requirements  based  on  precision,  or  variability  of  the  estimate. 
It  was  concluded  that  a  greater  degree  of  flexibility  and  coordination 
in  sampling  requirements  among  the  various  agencies  is  necessary  in  order 
to  further  facilitate  an  integrated  approach. 

Statisticians  agreed  on  the  importance  of  QC  sample  integrity  and  cost 
savings  in  integrated  QC  sample  designs.   States  can  elect  methods  for 
self-weighting  samples  or  more  complex  samples  requiring  weighted  error 
rates.   The  appropriate  approach  to  integrated  sampling  depends  upon  State 
circumstances  and  Federal  QC  sampling  requirements. 

Criteria  for  Submission  of  Integrated  QC  Sample  Design  Proposal 
Quality  Control  sample  integration  involves  a  variety  of  statistical, 
policy,  and  procedural  problems;  many  are  directly  related  to  the  States 
own  particular  situation.   As  a  result,  it  is  felt  that  each  State  wishing 
to  develop  an  integrated  sample  design  is  best  qualified  to  search  out 
solutions  within  the  frame-work  of  its  own  system.   Each  interested  State 
must  submit  its  proposal  concurrently  to  the  respective  Regional  Offices 
for  review.   The  first  and  foremost  criteria  for  approval  is  assurance  of 
sample  integrity  and  the  statistical  validity  of  the  sample  findings.   In 
addition,  the  impact  of  the  integrated  sample  design  upon  the  case  review 
process,  data  analysis,  reporting  requirements,  corrective  action  process, 
organization,  cost  and  staff  allocations  must  be  described. 
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Final  approval  at  the  Central  Office  level  is  required  prior  to  implemen- 
tation of  the  State's  sample  plan  in  order  to  insure  uniform  statistical 
standards  and  design  approvability. 

Future  Trends 

Although  the  various  Quality  Control  systems  provide  vital  data  on  the 
management  of  our  welfare  programs,  they  are  costly  and  we  need  to  look 
at  ways  of  making  them  as  cost  effective  as  possible.   It  is  anticipated 
that  States  will  become  increasingly  interested  in  QC  sample  integration 
as  a  mechanism  for  more  effective  resource  deployment.   In  order  to 
facilitate  this  approach,  it  is  recognized  that  a  greater  degree  of 
flexibility  and  coordination  is  required  at  the  Federal  level.   The  various 
Federal  agencies  must  carefully  evaluate  the  impact  of  sampling  and  program 
requirements  upon  integration.   Efforts  have  been  initiated  to  expedite  the 
approval  of  State's  proposals  for  integration,  as  well  as  coordinate  the  QC 
review  cycles,  and  develop  integrated  worksheets  and  materials. 

The  Federal  government  sees  its  role  as  one  of  joint  partnership  with  the 
States  in  the  effective  administration  of  the  welfare  programs,  and  is 
committed  to  providing  leadership  and  technical  assistance  to  States  in 
the  development  and  implementation  of  cost  saving  initiatives. 
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Section  II  -  Technical  Presentations 

Technical  experts  representing  the  various  programs 
were  asked  to  make  presentations  that  would  provide 
a  background  in  which  specific  aspects  of  QC  sample 
integration  may  be  viewed.   Results  of  Federal 
planning  efforts  and  recent  developments  in  the 
various  QC  programs  were  discussed  along  with  criteria 
for  submitting  proposals  for  integrated  sample  designs. 
In  addition,  presentations  were  made  on  the  various 
types  of  integrated  QC  sample  designs. 
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PLANNING  EFFORTS  IN  INTEGRATED  QUALITY  CONTROL 
SAMPLE  DESIGNS 


By:   David  Arnaudo,  Director 

Income  Maintenance  Evaluation 
Office  of  Research  and  Statistics 
Social  Security  Administration 
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Introduction 

The  Office  of  Research  and  Statistics  sponsored  planning  grants  to  New 
York,  South  Dakota,  and  Montana  to  examine  the  feasibility  of  integrating 
all  three  (AFDC,  Medicaid,  and  Food  Stamps)  Quality  Control  systems.   The 
objectives  were  to  develop  a  sampling  design  which  would  integrate  the 
samples  and  coordinate  the  review  procedures  of  the  three  QC  systems. 

Office  of  Research  and  Statistics  in  SSA  has,  in  addition,  sponsored  a 
grant  to  Colorado  to  demonstrate  the  operational  implementation  of  an 
integrated  design  for  the  three  QC  systems. 

Focus  of  Findings 

Thus  far  these  grants  have  called  attention  to  four  different  areas. 

1)  There  are  direct  benefits  that  would  result  from  the  integration  of 
QC  efforts,  as  follows: 

a)  savings  in  travel  time  to  and  from  interviews 

b)  savings  in  total  interviewing  time 

* 

c)  reduction  of  the  number  of  cases  reviewed 

d)  possible  improvement  of  quality  and  timeliness  of  completed 
reviews . 

2)  There  are  indirect  benefits  resulting  from  integrating  the  QC  systems 
as  follows: 

a)   the  elimination  of  some  forms 
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b)  automation  of  evaluation  systems 

c)  integration  of  corrective  action  planning 

d)  provision  of  data  on  multiple  program  participation 

e)  coordination  of  Federal  activities  regarding  QC . 

3)   In  considering  sample  integration,  attention  or  caution  must  be 
paid  to: 

a)  the  importance  of  multi-program  training  of  reviewers 

b)  the  necessity   for  accurate  and  flexible  monthly  sampling,  or 
the  use  of  appropriate  weighting 

c)  meeting  sampling  needs  and  time  demands  of  the  three  Federal 
agencies 

d)  rigorous  evaluations  of  methodology,  accuracy,  and  cost  benefits 

e)  avoiding  pitfalls  of  differing  definitions  among  programs,  which 
may  cause  duplications 

f)  careful  interagency  cost  allocation,  such  that  matchable  adminis- 
trative expenses  incurred  can  be  accurately  claimed  from  each 
program. 

^)   Federal  coordination  among  the  three  programs  is  needed  regarding: 

a)  timing  of  regulations  issuance 

b)  coordinating  time  frames  for  sampling  and  case  review 
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c)  uniform  definitions  (e.g.,  "household",  "case",  etc.) 

d)  flexibility  with  regard  to  waivers  for  further  experimentation, 
to  obtain  maximum  results 

e)  integration  at  the  policy  level. 
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DEVELOPMENTS  IN  THE  AFDC  QC  PROGRAM  - 
IMPLICATIONS  FOR  INTEGRATION 


BY:   John  X.  Bowes,  Director 

Division  of  Quality  Control 

Management 
Social  Security  Administration 
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We  at  the  Federal  level  recognize  and  appreciate  the  additional  burdens 
being  placed  on  States  by  the  various  QC  systems.   Although  these  systems 
provide  vital  data  on  the  management  of  our  welfare  programs,  they  are 
also  costly  systems  and  we  need  to  look  for  ways  of  making  them  as  cost 
effective  as  possible.   We  believe  that  the  many  similarities  among  the 
QC  systems,  and  the  increased  overlap  in  the  populations  reviewed,  provides 
an  excellent  opportunity  for  States  to  achieve  savings  through  integration. 
As  a  word  of  caution,  however,  it  must  be  recognized  that  QC  sample  inte- 
gration is  a  complicated  procedure,  with  many  variables  which  must  be  taken 
into  consideration  before  deciding  the  possible  advantages  and  disadvantages 
it  may  hold.   Many  of  these  variables  are  determined  by  the  State's  own 
particular  situation  and  as  a  result,  each  State  must  decide  if  integration 
is  advantageous  to  them.   We  are  hopeful  that  this  conference  will  assist 
States  in  reaching  this  decision.   With  these  few  words  in  mind,  I  would 
like  to  pass  along  to  you  some  recent  developments  in  the  AFDC-QC  program 
which  may  have  implications  for  sample  integration. 

Fiscal  Incentives 

As  you  are  aware,  the  fiscal  incentive  provision  extracted  from  House 
Bill  7200  was  recently  enacted  into  law.   Under  this  provision.  States 
reaching  a  "dollar  error  rate"  of  less  than  4  per  centum  may  be  eligible 
for  a  share  of  the  Federal  savings.   As  the  law  reads,  however,  the  "dollar 
error  rate"  would  be  the  composite  dollar  error  rate  for  active  cases 
( ineligibles,  overpayments,  and  underpayments)  and  negative  cases  (erroneous 
denials  and  terminations).   The  current  negative  QC  system  does  not  accommo- 
date dollar  error  rate  findings.   To  do  so  would  require  Quality  Control  to 
determine  eligibility  and  payment  amounts  for  negative  cases  not  correctly 
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denied  or  terminated  for  the  reason  given  by  the  local  agency.   This 
issue  remains  to  be  resolved  at  the  present  time. 

H.  R.  7200 

H.  R.  7200  itself  was  passed  by  the  House  of  Representatives  on  June  lA, 
1977  and  is  currently  under  consideration  by  the  Senate.   Although  we 
believe  that  some  of  its  provisions  would  be  beneficial  to  the  AFDC-QC 
program,  the  Department  also  feels  that  some  of  the  provisions  will  not 
produce  useful  results  and  we  are  attempting  to  have  these  changed.   In 
either  event,  the  major  provisions  of  H.  R.  7200  as  now  constituted  are 
as  follows: 

-  would  put  QC  into  law. 

-  would  establish  performance  evaluation  and  corrective  action  by 
"operating  units".   We  are  not  sure  what  is  meant  by  the  term 
operating  units  and  whether  any  increase  in  sample  sizes  would 
be  necessitated. 

-  would  identify  where  and  by  whom  cases  were  handled  and  the 
dates  of  completed  or  due  reconsiderations. 

-  would  require  States  to  submit  their  case  review  findings  no 
later  than  60  days  following  the  sample  month  in  which  the 
case  was  selected.   Under  this  requirement,  any  integrated 
design  would  have  to  provide  for  timely  and  independent  AFDC-QC 
case  findings . 

-  would  require  Corrective  Action  reports  to  be  submitted  no  later 
than  120  days  after  the  close  of  the  review  period. 
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-  would  require  a  review  of  State  actions  taken  to  correct  individual 
error  cases  in  the  previous  six-month  period. 

-  would  require  a  full  field  investigation  in  all  Federal  re-review 
cases. 

-  would  require  that  the  Federal  government  complete  the  QC  sample 
and  bill  the  State  twice  the  cost  if  a  State  fails  to  comply  with 
QC  requirements. 

-  would  provide  for  a  greater  oversight  of  the  QC  system  by  the 
Inspector  General's  office. 

Change  in  QC  Review  Cycle 

Another  recent  development  which  has  an  impact  on  integrated  sample  designs 
is  the  shift  from  a  calendar  to  a  fiscal  year  AFDC-QC  review  cycle. 
Although  this  shift  puts  AFDC-QC  on  the  same  cycle  as  MEQC ,  it  puts  AFDC-QC 
out  of  phase  with  FS-QC .   We  believe  having  all  three  QC  systems  on  the  same 
review  cycle  is  essential  to  integration. 

Centralized  QC  Data  Base 

A  major  initiative  that  we  are  undertaking  at  the  Federal  level  is  the 
establishment  of  a  centralized  QC  data  base.   We  believe  that  such  a 
data  base  is  essential  to  our  goal  of  increased  technical  assistance  to 
the  States  in  the  determination  of  the  principal  causes  of  errors  and  in 
the  development  of  error  reduction  programs.   Our  plans  call  for  processing 
State  data  through  error  prone  profile  systems  and  supplying  the  States 
with  this  analysis.   To  this  end  the  Department  is  planning  to  propose  a 
regulation  which  would  require  States  to  submit  QC  case  review  findings  no 
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later  than  60  days  following  the  sample  month.   In  addition  to  insuring 
the  timeliness  and  consistency  of  State  data,  the  proposed  regulation  would 
eliminate  all  statistical  reporting  tables  now  required  from  the  States. 
Instead,  the  Federal  government  would  have  the  capacity  to  generate  the 
required  statistical  tables  as  well  as  error  identification  and  reduction 
goals,  error  prone  profiles,  and  other  management  reports  for  the  States. 

Re-evaluation  of  the  QC  Sample  Design 

The  last  major  development  in  the  AFDC-QC  program  in  process  is  a  re-exami- 
nation of  the  QC  sample  design.   We  are  considering  replacing  the  current 
step  approach  of  establishing  required  State  sample  sizes  with  a  formula 
approach.   This  method  would  be  directly  related  to  each  State's  actual 
caseload  and  would  eliminate  the  abrupt  changes  in  QC  workloads  as  sample 
sizes  increase  from  one  step  to  the  next  under  the  current  system.   Nationally, 
we  estimate  that  a  reduction  of  approximately  ^,450  required  active  review 
completions  can  be  realized  under  the  new  method. 

In  summary,  I  would  like  to  say  that  the  Department  sees  its  role  as  one 
of  joint  partnership  with  the  States  in  the  effective  administration  of 
the  AFDC  Program.   We  realize  that  a  greater  degree  of  coordination  among 
the  various  Federal  Agencies  must  take  place  if  such  efforts  as  QC  sample 
integration  are  to  be  facilitated.   The  Department  is  committed  to  assisting 
States  where  it  can  and  we  trust  that  this  conference  is  a  step  in  that 
direction.   Thank  You. 


18 


DEVELOPMENT  IN  THE  FS-QC  PROGRAM  - 
IMPLICATIONS  FOR  INTEGRATION 


By:   Carolyn  Merk,  Acting  Director 
Program  Development  Division 
Department  of  Agriculture 
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NATIONAL  CONFERENCE  ON  INTEGRATED  QUALITY  CONTROL  SYSTEMS 

I  am  going  to  talk  to  you  this  morning  about  some  past  history 
of  the  Food  Stamp  Program,  much  of  it  only  very  recently  past, 
and  about  new  directions  being  taken  in  food  stamps  that  will  affect 
dramatically  the  distribution  of  program  benefits,  in  terms  of  who 
is  eligible  and  for  how  much,  as  well  as  the  process  through  which 
benefits  are  distributed.   These  factors  are,  of  course,  the  core  of 
our  mutual  concern,  which  is  the  measurement  and  documentation  of 
the  extent  to  v/hich  program  benefits  are  distributed,  in  the  correct 
amounts,  as  intended  by  legislation  and  regulation,  to  the  appropriate 
households . 

We  In  the  Department  of  Agriculture  are  currently  planning  for  the 
implementation  of  new  food  stamp  legislation,  legislation  that  is 
labeled  "reform".   Reform  has  been  a  popular  term  to  use  with  regard 
to  our  V7elfare  programs  for  over  ten  years  now,  and,  usually,  it  is 
brought  up  when  a  program  or  programs  that  frequently  experienced 
considerable  popularity  during  their  early  years — programs  that 
carry  an  entitlement  of  some  subset  of  the  population  to  the  tax 
dollars  of  others  within  the  population — grow  to  the  point  at  which 
either  their  costs,  the  number  of  recipients,  or  both,  begin  to  claim 
large  amounts  of  tax  dollars  or  to  cover  perhaps  disturbingly  large 
numbers  of  people. 

To  a  major  extent,  the  quality  control  initiatives  in  welfare  programs 
have  been  brought  about  in  response  to  the  same  phenomena  as  reform 
initiatives,  that  is,  large  numbers  of  dollars  going  to  large  numbers 
of  people  through  a  complex  system.   And,  in  any  situation  of  this 
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kind,  things  are  going  to  go  wrong  sometime,  and  money  will  be 
wrongly  spent.   The  occurrence  of  mistakes  and  the  resulting  waste, 
almost  regardless  of  how  infrequent  or  small,  can  result  in  the 
undermining  of  the  entire  system.   Therefore,  it  is  important  to 
accurately  measure  and  interpret  systematic  errors  so  that  appropriate 
corrections  can  be  made. 

In  the  mid-1960 's,  HEW  started  development  of  a  system  which  used 
modern  industrial  quality  control  principles  to  measure  the  quality 
of  the  welfare  delivery  system.   Later,  with  the  vast  growth  in 
costs  and  caseloads  under  the  Food  Stamp  Program,  USDA  began  to 
adapt  HEW's  system  for  AFDC  to  its  own  program.   The  food  stamp 
program  regulations  now  require  a  quality  control  system  in  each 
State.  '  The  system  is  an  ongoing  statistically  valid  review  of  parti- 
cipating households:  to  validate  the  accuracy  of  the  household's 
eligibility,  to  determine  the  extent  to  which  households  are  paying 
the  proper  purchase  requirements  and  receiving  the  coupon  allotments 
and  therefore  the  benefits  to  which  they  are  entitled,  and  to  insure 
that  decisions  to  deny  or  terminate  a  household's  participation  are 
valid. 

At  first,  the  food  stamp  quality  control  system  covered  only  non- 
assistance  households — that  is,  those  households  in  which  no  member 
received  welfare,  or  only  some,  but  not  all  members,  received 
welfare.   Public  assistance  households — those  in  which  all  members 
are  receiving  welfare — were  not  covered  by  QC  at  first,  partly 
because  it  was  believed  that  HEW  could  cover  these  cases  through 
their  own  AFDC  quality  control  system.   This  made  a  lot  of  sense  at 
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the  beginning  of  food  stamp's  quality  control  because  welfare  households 
were  categorically  eligible  for  food  stamps;  that  is,  if  all  house- 
hold members  were  included  in  the  grant,  the  household  did  not  have 
to  meet  the  food  stamp  income  and  asset  tests. 

However,  as  the  food  stamp  program  grew  larger  and  more  complex, 
as  States  became  more  experienced  in  the  food  stamp  quality  control 
process,  and  the  elimination  of  categorical  eligibility  for  AFDC 
and  SSI  recipients  became  more  of  an  issue  in  food  stamps,  it  became 
necessary  to  start  including  public  assistance  cases  in  food  stamp 
quality  control  reviews.   This  meant,  of  course,  that  some  households 
were  included  twice  in  quality  control  reviews — for  HEW  and  for  USDA. 
This  is  the  point  at  which  integrated  sampling  has  come  into  our 
individual  quality  control  systems.   Where  the  HEW  and  USDA  sampling 
universes  overlap,  samples  may  be  drawn  simultaneously,  and  even 
personal  interviews  may,  in  some  situations,  be  conducted  for  both 
programs  at  the  same  time.   For  those  of  you  interested  in  integrated 
sampling  in  these  programs,  it  would  be  helpful  for  you  to  know  how 
the  recent  food  stamp  changes  will  affect  the  kinds  of  things  that 
QC  reviews  must  cover  as  well  as  our  best  estimates  of  how  the  "error 
proneness"  of  the  food  stamp  caseload  will  change. 

As  you've  probably  read  in  the  newspapers.  President  Carter  signed 
the  Food  Stamp  Act  of  19  77  into  law  on  September  29,  1977.   In  many 
ways,  these  are  the  most  sweeping  changes  to  the  Food  Stamp  Act 
since  its  inception  in  1964  when  we  received  our  mandate  to  operate 
as  a  full-fledged  Federal  program — not  just  a  pilot  program. 
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Probably  the  most  important,  and  certainly  the  most  controversial 
change  of  the  new  legislation  is  the  elimination  of  the  food  stamp 
purchase  requirement.   At  the  present  time,  participants  must  pay  for 
their  stamps,  an  amount  that  varies  by  their  countable  income  and 
family  size.   Countable  income  is  gross  monthly  income  after  a  long 
series  of  deductible  expenses  are  subtracted.   It  is  in  the  income 
determination  and  the  deduction  determination  process  where  many 
food  stamp  errors  are  made,  and  the  new  legislation  centers  on  these 
■factors. 

The  new  food  stamp  legislation  will  simplify  the  income  and  deduction 
determination  process  because  it  replaces  a  series  of  eight  allowable 
deductions  with  only  three.  Under  current  rules,  a  caseworker  must 
compute,  and  a  quality  control  worker  must  verify,  a  work  allowance 
deduction;  mandatory  deductions  from  earned  income  such  as  local. 
State  or  Federal  income  tax  withholding  payments.  Social  Security 
taxes,  mandatory  retirement  payments,  and  mandatory  union  dues;  a 
medical  expense  deduction;  a  child  care  deduction  for  workers;  a 
tuition  and  mandatory  school  fees  deductions;  a  court  order  support 
and  alimony  payments  deduction;  unusual  expense  or  casualty  losses 
deduction;  and  a  shelter  cost  deduction. 

These  deductions  can  be  very  difficult  for  a  quality  control  worker 
to  verify.   For  example,  in  the  medical  expense  deduction,  the  reviewer 
may  have  to  verify  the  amount  of  health  and  hospitalization  insurance 
premiums  paid,  including  medicare  payments.   He  may  also  have  to 
verify  the  amount  spent  for  prescribed  drugs,  the  existence  and  neces- 
sity of  prosthetic  devices  or  even  a  seeing-eye  dog  in  the  household 
and  costs  associated  with  it.   Then  the  reviewer  must  be  sure  to  learn 
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whether  or  not  any  of  the  medical  expenses  paid  out  have  been 
reimbursed  through  insurance.  And  the  reimbursement  must  then  be 
put  in  the  proper  place  on  the  quality  control  form  as  a  lump  sum 
payment. 

Under  our  new  legislation,  there  will  be  only  three  deductions— a 
standard  deduction  of  $60  a  month  for  each  household,  a  20  percent 
of  earned  income  deduction  for  households  with  earnings,  and  a  com- 
bined shelter/child-care  deduction  with  a  maximum  of  $75. 

This  change  should  lead  to  fewer  errors  in  the  food  stamp 
certification  process  as  well  as  simplify  the  work  of  the  average 
State  quality  control  reviewer. 

I  mentioned  before  that  the  purchase  requirement  would  be  eliminated 
under  our  new  legislation.   This  does  not  mean,  as  reported  in  some 
newspapers,  that  food  stamp  recipients  will  simply  get  the  same 
amount  of  stamps  they  get  now— but  without  paying  for  them.   What 
it  means  is  they  vd.ll  continue  to  receive  the  bonus — the  same  amount 
that  is  supplied  now  by  the  Federal  Government,  and  they  will  no 
longer  have  to  turn  over  to  the  Government  a  portion  of  their  ovm 
money  to  be  converted  into  food  stamps.   This  may  also  simplify  the 
certification  and  quality  control  processes,  but  it  is  too  early  to 
tell. 

There  is  another  change  in  our  new  legislation  which  should  simplify 

our  quality  control  process.   Households  will  no  longer  have  to  have 

access  to  cooking  facilities,  nor  constitute  the  difficult  to 

verify  "economic  unit"  thereby  eliminating  another  source  of  quality 
control  errors. 
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The  final  important  change  impacting  on  quality  control  is,  as  I 
mentioned  before,  the  elimination  of  the  provision  for  allowing 
public  assistance  households  to  be  categorically  eligible  for 
food  stamps  without  meeting  an  income  and  an  asset  test.   Thus, 
some  households  who  are  receiving  AFDC  and  SSI  now  will  no  longer 
be  eligible  for  food  stamps. 

Why  V7ere  these  changes  made  in  the  income  formula  of  the  Food 
Stamp  Program?   It  wasn't  only  to  simplify  the  quality  control 
process,  although  we  hope  that  may  be  a  result.   The  new  law 
accomplishes  the  redistribution  of  Program  benefits  from  the  not  so 
poor  to  the  poorest  of  the  poor.   We  had  discovered  that  a  very 
unfortunate  result  of  our  previous  system  of  allowing  the  eight 
allowable  deductions  was  that  higher  income  households  were  quali- 
fying for  Program  benefits  because  they  could  afford  the  kind  of 
expenses  that  qualified  as  deductions,  while  other  households  with 
the  same  incomes  and  compelling  expenses,  which  didn't  happen  to 
meet  our  deduction  requirements,  were  being  denied  the  benefits  of 
the  Program.   That  is  why  the  new  legislation  did  away  with  the 
Itemized  deductions  and  replaced  them  with  the  standard  and  other 
two  deductions  and  replaced  the  variable  benefit  reduction  rate  of 
the  basis  of  issuance  tables  with  a  uniform  rate  applying  equitably 
to  all  households.   Another  important  change  which  accomplished  the 
limitations    of  Program  benefits  to  the  poorest  households  was 
the  establishment  of  the  net  income  eligibility  limits  for  the 
program  at  the  0MB  poverty  level,  currently  $5,850  a  year  for  a 
four-person  household. 
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Finally,  the  food  stamp  asset  test  was  broadened  for  the  first 
time  to  allow  households  to  retain  $1,750  in  liquid  assets,  rather 
than  only  $1,500  as  most  of  them  are  allowed  now. 

The  law  also  contains  some  interesting  funding  provisions  of  parti- 
cular interest  to  you  as  State  officials,  although  it  is  unknown 
yet  exactly  how  these  provisions  will  affect  quality  control.   One 
provision  involves  sanctions.   Federal  funds  for  administrative 
costs  will  be  withheld  from  States  failing  to  meet  program  standards 
without  good  cause.   A  State  may  be  required  to  pay  for  food  stamps 
improperly  issued  if  the  State  has  been  negligent  or  has  committed 

* 

fraud.   USDA  will  no  longer  have  to  establish  a  gross  negligence 
charge  before  billing  a  State. 

Another  important  funding  provision  involves  incentives.   The  Federal 
Government  will  pay  an  additional  10  percent  of  States'  administrative 
costs  if  they  reduce  their  overall  bonus  dollar  error  rates  to  below 
five  percent.   In  addition,  these  States  will  not  have  to  submit 
corrective  action  plans  relating  to  error  rates.   And  the  final  new 
and  interesting  funding  provision  involves  fraud.   The  new  law 
increases  Federal  reimbursements  to  States  from  50  percent  to  a 
minimum  of  75  percent  of  costs  incurred  in  investigating  and  prose- 
cuting fraud.   People  found  guilty  of  fraud  by  an  administrative 
hearing  will  be  dropped  from  the  Program  for  3  months.   People  found 
guilty  by  the  courts  will  be  dropped  from  6  to  2A  months. 
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With  all  of  .these  changes,  what  kind  of  people  can  we  expect  to  be 
participating  in  the  Food  Stamp  Program  once  the  new  legislation  is 
implemented?  What  can  you  expect  as  quality  control  officials? 
First,  the  size  of  the  participating  caseload  will  increase — from 
an  estimated  5.4  million  households  projected  for  the  current  program 
in  FY  1978  to  an  estimated  6.1  million  households  once  the  new 
legislation  is  fully  implemented.   At  the  same  time,  the  size  of  the 
potentially  eligible  universe  will  decline.   About  half  of  the  current 
caseload — 47.3  percent — should  expect  no  change  in  their  food  stamp 
bonus  as  a  result  of  the  new  legislation.   Nearly  a  quarter  of  the 
current  caseload — 24.2  percent — will  have  their  bonus  increased  under 
the  new  legislation.   The  households  with  increased  bonuses  are 
primarily  those  who  have  deductions  so  small  now  that  they  will  bene- 
fit from  the  $60  standard  deduction.   About  one-fifth  of  the  current 
caseload  will  remain  eligible  for  Program  benefits,  but  because  they 
currently  are  taking  high  deductions,  they  will  have  their  bonuses 
reduced  under  the  new  legislation.   Finally,  nearly  a  tenth  of  the 
current  caseload  will  become  ineligible  because  their  net  incomes 
will  exceed  the  poverty  line.   Thus,  the  people  ^^;ho  will  be  losing 
program  benefits  are  primarily  the  high  income,  high  deduction  people— 
those  who  are  prone  to  quality  control  errors. 

We  can  only  speculate  on  the  estimated  3  million  people  who  will 
enter  the  Program  for  the  first  time  because  of  the  elimination  of 
the  purchase  requirement.   We  think  they  will  be  households  with 
earned  income,  because  we  estimate  that  most  of  the  households  who 
receive  some  form  of  welfare  income  are  already  participating  at 
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fairly  high  rates.  We  have  estimated  further  that  most  of  these 
households  with  earned  income  are  larger  than  average  households- 
because  our  experience  indicates  that  households  with  earners  are 
usually  households  with  a  male  head,  which  generally  means  at  least 
one  additional  person  per  households.   Unfortunately,  these  kinds 
of  households  may  be  more  error  prone.   But  at  the  same  time,  they 
usually  are  not  eligible  for  larger  than  average  bonuses  because 
of  their  earned  income. 

Then,  there  are  about  600,000  households  who  will  enter  the  Program 
because  of  the  increased  asset  screen.   They  may  well  be  elderly, 
stable  income  households,  because  we  know  that  the  elderly  have 
more  liquid  assets  than  younger  households. 

In  sum  there  will  be  many  interesting  changes  taking  place  in  the 
-food  stamp  caseload  and  in  the  quality  control  system  as  a  result 
of  our  new  legislation.   The  job  for  food  stamp  quality  control 
reviewers  should  in  many  ways  be  easier  because  the  new  law  has  • 
done  much  to  make  the  program  simplier  to  understand  and  administer. 
However,  we  in  the  Department  of  Agriculture  still  have  some  very 
important  decisions  to  make  about  quality  control  in  the  coming 
weeks,  mile   I  wish  I  could  share  more  specific  QC  plans  with  you, 
I  cannot,  primarily  because  we  have  no  firm  plans  yet.   In  fact,  we 
are  still  drafting  portions  of  the  regulations  that  the  QC  system  will 
cover,  and  then,  of  course,  the  draft  regulations  will  be  made  available 
for  public  comment  before  final  publication. 
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However,  I  can  say  that  our  commitment  to  pursuing  integrated 
sampling  is  strong,  our  commitment  to  an  efficient  and  rational 
system  is  strong,  and,  simultaneously,  we  are  committed  to  ensuring 
that  the  objectives  of  the  new  legislation  are  met. 
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DEVELOPMENTS  IN  THE  MEQC  PROGRAM  - 
IMPLICATIONS  FOR  INTEGRATION 


By:   Allan  Lazar,  Acting  Chief 
Management  Performance 

and  Statistical  Analysis  Branch 
Office  of  Financial  Management 
Health  Care  Financing  Administration 
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As  most  of  you  are  aware,  the  Health  Care  Financing  Administration 
is  currently  in  the  process  of  implementing  a  major  redesign  of  the 
Medicaid  Quality  Control  (MQC)  system.   We  believe  that  the  new  MQC 
system,  to  be  effective  April  1,  1978,  will  correct  the  major  deficiencies 
of  the  previous  system  and  more  importantly  for  the  purposes  of  this 
conference,  have  an  important  influence  upon  States  interest  in  integrated 
sample  designs  as  a  management  alternative.   Under  the  redesign,  the  MQC  sample 
universe  has  been  expanded  to  include  the  total  Medicaid  population  and 
although  efforts  have  been  made  to  minimize  State  QC  staff  requirements, 
sample  sizes  will  increase. 

Most  important  to  integrated  sample  designs,  however,  is  the  switch  in 
sampling  from  paid  claims  to  sampling  from  the  case  eligibility  files; 
just  as  in  AFDC  and  Food  Stamp  QC .   Prior  to  this  switch  MQC  did  not 
readily  lend  itself  to  sample  integration  because  Medicaid's  sample 
frame  of  paid  claims  was  inconsistent  with  AFDC  and  Food  Stamp's  case 
sample  selection.   Cases  selected  from  paid  claims  were  reviewed  in 
relation  to  the  time  frame  in  which  the  service  was  provided.   The 
review  date  did  not  correspond,  and  may  have  been  up  to  nine  months 
prior  to  the  selection  date.   Now,  however,  the  case  file  universe  will 
be  sampled  for  each  QC  system  and  cases  will  be  reviewed  for  eligibility 
as  of  the  sample  month.   As  a  result,  any  case  selected  in  a  3-way  (AFDC/ 
MQC/FSQC)  integrated  sample  can  be  simultaniously  reviewed  for  each  program 
benefit  being  received  and  more  than  ever  before,  interested  States  have 
the  opportunity  to  realize  the  full  potential  savings  associated  with 
integrated  sample  designs. 
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I  would  now  like  to  provide  you  with  a  brief  overview  of  the  revised 
Medicaid  Quality  Control  System.   The  past  problems  and  deficiencies 
will  be  outlined  as  well  as  the  key  revisions  in  sampling  and  review 
requirements.   Although  we  realize  that  there  will  be  difficult  times 
ahead,  we  believe  that  the  revised  MQC  system  will  provide  States  with 
the  type  of  data  necessary  to  take  meaningful  and  effective  corrective 
actions.   An  overview  of  the  key  points  are  to  be  found  on  the  pages 
follow. ) 
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THE  PROBLEM 


Problem  I  -  Defective  Medicaic  Program 

-  imprecise  error  estimates 

-  difficult  to  check  eligibility 

-  outdated  error  findings 

-  partial  coverage 

Problem  II  -  Big  Dollar  Losses  (FY'77  est.) 

-  $1,2  billion  misspent  on  ineligibles 

-  $600  million  plus  on  unrecovered  third  party  insurance  (TPL) 

-  $200  million  on  claims  processing  (CP)  errors 

Problem  III  -  No  adequate  measurement  system 
'-  Old  QC  faulty 

-  nothing  for  TPL,  CP  errors 

-  "flying  blind" 


THE  SOLUTION 


Revised  Medicaid  QC  Program 

-  Broaden  sample  to  total  Medicaid  population 

-  Switch  to  current  month  case  sampling 

-  Add  TPL,  CP  review 

Benef i  ts 

-  Measure  true  level  of  errors,  dollar  loss 

-  Ineligibility 

-  TPL 

-  CP 

-  More  accurate,  current  data  on  error  causes 

-  Better  basis  for  effective  Corrective  Action 

-  Compatible  with  AFDC,  SSI,  FS  QC 


REVISED  MEDICAID  QC 


Claims  to  case  sampling 


Coverage  of  total  Medicaid  Population 


Addition  of  TPL  and  Claims  Processing  error  measurement 


Integration  with  AFDC-QC,  SSI-QA 


en 


Implementation  Scheduled  April  1,  1978 


125  Positions  for  QC  and  TA 


REVISED  MEDICAID  QUALITY  CONTROL  (MQC)  SAMPLE 


1 

, 

AFOC-qC  SampU 
43,000 

Medicaid  Beneficiaries 

SSI-QA  Sample 
17.000  cases 

•■ 

~^ 

1 

AFDC  Bene-  •  Medically  Needy 

SSI  Bene- 
ficiaries 

■' 

ficlarles  ; 

Current  Medicaid 
QC  Sample 
17.500  Cases 

i 

:  . 

' 

AFDC 

;  Medically  1 

eedy     j  SSI 

Revised  Medicaid  QC  Sample 
77,500  Cases 

1 , .                . 

t                                           • 

1   1   — 

Revised  MQC  Sample  •  77,500  (MEQC  17,500  +  AFDC  43.000  +  SSI  17,000). 

Operation        <•  will  be  Implemented  April -September  '78 
Features        -  QC  coverage  of  total  Medicaid  population;  avoids 

duplicative  QC  reviews;  ninimizes  State  QC  staff 

requirements. 


INTEGRATED  SAMPLE  DESIGN  METHODS 


By:   Robert  Schneider 

Private  Consultant 
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HOW  TO  DO  IT 

INTEGRATED  SAMPLE  SELECTION 

FOR  AFDC,  FOOD  STAMP,  AND 

MEDICAID  QUALITY  CONTROL 
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PREFACE 

In  each  state  there  is  a  single  agency  responsible 
for  the  administration  of  the  AFDC  program,  the  Food 
Stamp  program  and  the  Medicaid  program.   These  agencies 
are  required  by  the  Federal  Government  to  operate  a  quality 
control  system  for  each  of  the  three  programs.   The  purpose 
of  the  quality  control  systems  is  to  measure  and  reduce  the 
frequency  of  benefit  errors,  which  are  benefits  disbursed 
to  or  for  ineligible  recipients  or  incorrect  benefit  amounts 
for  eligible  recipients. 

The  three  quality  control  systems  have  many  similarities. 
Each  is  based  on  a  monthly  random  sample  of  program  benefits 
(AFDC  payments,  Food  Stamps,  or  medical  bills  paid).   The  QC 
sample  cases  are  investigated  very  thoroughly  to  determine 
the  eligibility  of  the  recipient  and  the  correctness  of  the 
benefit  amount.   The  QC  review  also  collects  information 
about  the  causes  of  any  errors  found.   This  information  is 
used  for  corrective  action  planning  to  eliminate  the  causes  of 
benefit  errors. 

In  the  past,  the  three  QC  systems  have  been  operated 
pretty  independently.   Each  system  had  its  own  sample 
(unrelated  to  the  other  system  samples)  and  when  a  case  was 
reviewed,  only  the  benefit  which  caused  the  case  to  be  selected 
was  reviewed  even  though  the  same  individuals  might  have  also 
been  receiving  benefits  from  another  program.   In  some  states, 
individual  QC  reviewers  conducted  only  one  type  of  review 
(AFDC,  Food  Stamp  or  Medicaid). 
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Until  recently,  this  was  a  reasonable  approach,  since 
there  was  not  much  overlap  in  the  universes  sampled  from 
for  the  three  QC  systems.   Until  July  1976,  AFDC  cases 
receiving  food  stamps  were  excluded  from  the  Food  Stamp 
quality  control  system.   Thus,  there  was  no  overlap  in  the 
universes  sampled  from  for  the  AFDC  and  Food  Stamp  quality 
control  systems.   The  Food  Stamp  QC  system  has  now  been 
expanded  to  include  AFDC  cases  receiving  Food  Stamps.   The 
Medicaid  Eligibility  QC  system  has  not,  but  is  now  being 
modified  to  include  AFDC  cases  (as  well  as  some  other  types 
of  cases  which  were  previously  excluded,  e.g.  SSI  cases). 
The  MEQC  system  is  also  being  modified  to  use  a  sample  of 
eligible  cases  rather  than  a  sample  of  paid  claims.   Figure  1 
shows  the  relationship  which  will  exist  between  the  universes 
for  the  three  QC  systems. 


Figure  1.   Overlapping  universes  for  AFDC, 
Food  Stamp,  and  Medicaid  QC . 
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Many  families  receive  more  than  one  of  the  benefits 
being  considered  here.   There  would  be  a  considerable  degree 
of  efficiency  involved  if  one  family  could  be  included  in 
more  than  one  QC  sample.   There  would  be  only  one  home  visit 
and  travel  is  a  significant  cost  in  QC.   Many  of  the  items 
(e.g.  assets  and  income)  which  have  to  be  verified  pertain 
to  all  three  programs.   For  these  reasons,  and  other  reasons 
discussed  later,  the  cost  of  doing  two  reviews  in  one  case 
is  significantly  less  than  the  cost  of  doing  two  reviews 
in  two  different  cases. 

The  term  integrated  sampling  has  been  adopted  to 
describe  various  methods  whereby  cases  selected  for  one  QC 
sample  can  also  be  utilized  as  part  of  one  (or  two)  of  the 
other  QC  samples.   Several  different  approaches  are  described 
in  this  guide.   The  differences  among  the  various  approaches 
are  in 

1)  The  conditions  required  (e.g.  some  approaches  require 
that  the  benefit  combination  in  each  case  must  be 
known  while  the  sample  is  being  selected.   The  other 
approaches  do  not  require  that  the  benefit  combination 
be  known  at  the  time  of  sample  selection) . 

2)  The  end  result  (some  approaches  produce  self -^weighting 
samples  while  the  other  approaches  produce  stratified 
samples  which  must  be  combined  with  weighting  factors) . 

3)  The  relative  efficiency  (some  approaches  are  less  efficient 
because  less  multiple  benefit  cases  are  selected  or  not 

all  of  the  benefits  are  reviewed  in  some  of  the  multiple 

benefit  cases)  , 
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For  the  full  potential  efficiencies  of  integrated 
sampling  to  be  realized,  QC  reviewers  must  be  able  to  complete 
any  type  of  QC  review.   There  are  economies  involved  in  having 
generalized,  or  generic,  QC  reviewers  even  if  integrated 
sample  selection  is  not  used.   Since  generic  reviewers  can 
perform  more  functions,  each  can  be  assigned  to  a  smaller 
geographical  area  which  reduces  the  total  time  and  money 
spent  travelling.   Also,  having  cases  to  be  reviewed  located 
closer  together  reduces  time  spent  waiting  if  time  that  would 
otherwise  be  spent  waiting  on  a  case  can  be  used  to  do  some- 
thing on  another  case. 

Information  from  states  that  have  trained  specialized 
reviewers  to  be  generic  indicates  that  this  is  not  overly 
difficult  to  do  and  does  not  result  in  any  reduction  in  the 
quality  of  reviews.   What  is  involved  is  about  a  weeks  worth 
of  training  and  some  extra  supervision  for  the  following 
couple  of  months.   More  information  on  the  integrated  review 
process  and  the  economies  involved  is  contained  in  An  Evaluation 
of  Integrated  Sample  Selection  for  AFDC  and  Food  Stamp  Quality 
Control.   The  conclusion  reached  there  is  that  the  percentage 
reduction  in  QC  review  costs  that  can  be  achieved  with  generic 
reviewers  and  integrated  sample  selection  is  at  lease  as  large 
as  the  percentage  reduction  in  the  number  of  distinct  cases  to 
be  reviewed.   The  reason  for  this  is  that  the  extra  work  in 
multiple  benefit  reviews  is  more  than  offset  by  the  other 
economies  involved  in  having  generic  reviewers. 
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With  some  of  the  integrated  sampling  procedures  described 
later,  the  number  of  distinct  cases  to  be  reviewed  can  be 
reduced  by  about  50%,  compared  to  the  number  of  cases  that 
would  have  to  be  reviewed  if  completely  independent  samples 
were  used  for  AFDC,  Medicaid,  and  Food  Stamp  Quality  Control. 

Since  50%  of  the  QC  active  case  review  costs  is  a  large 
amount  of  money  on  a  national  basis,  it  appears  that  integrated 
sample  selection  should  be  pursued  wherever  possible  in  spite 
of  the  additional  complications  involved. 
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INTEGRATED  SAMPLE  SELECTION 
FOR  MEDICAID  AND  OTHER 
QUALITY  CONTROL  SYSTEMS 
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I.   INTRODUCTION 


The  expanded  Medicaid  Quality  Control  system  is  designed 
to  provide  much  more  useful  information  than  the  previous 
MEQC  system,  which  used  relatively  small  samples  compared  to 
other  QC  systems  and  covered  only  part  of  the  total  population 
of  Medicaid  recipients.   The  expanded  Medicaid  QC  sample  sizes 
are  several  times  as  large  as  the  previous  MEQC  sample  sizes 
in  most  states.   However,  by  integrating  the  Medicaid  quality 
control  sample  selection  with  samples  from  other  QC  systems, 
the  expanded  Medicaid  QC  system  can  be  implemented  with  only 
a  small  increase  or  even  a  decrease  in  the  total  QC  workload 
in  a  state. 

Two  approaches  can  be  used  in  selecting  the  Medicaid 
quality  control  sample  cases.  One  approach  is  to  use  all 
of  the  AFDC  quality  control  cases,  all  of  the  SSI  quality 
assurance  cases,  and  an  additional  sample  of  Medical  Only 
cases  (the  same  number  that  were  selected  in  the  previous 
MEQC  system) .  With  this  approach  there  is  a  small  increase 
in  the  number  of  field  investigations  (those  required  to 
determine  if  ineligible  SSI  cases  would  still  be  eligible 
for  Medicaid) . 

The  other  approach  is  to  integrate  Medicaid  QC  sample 
selection  with  QC  sample  selection  for  AFDC  quality  control 
and  Food  Stamp  quality  control.   The  primary  advantage  of 
this  approach  is  that  the  increased  Medicaid  QC  sample  sizes 
can  be  obtained  and  a  significant  reduction  in  the  total  QC 
workload  can  be  realized  at  the  same  time. 
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There  are  many  different  sampling  procedures  which  could 
be  used  to  obtain  integrated  samples  for  Medicaid,  AFDC,  and 
Food  Stamp  Quality  Control.   Five  methods  are  described  here. 
All  of  these  methods  are  discussed  in  more  detail  elsewhere. 
The  purpose  of  this  summary  is  to  provide  a  general  description 
of  each  method  and  some  discussion  of  the  complications  involved 
and  the  relative  efficiency  of  each  method.   With  this  infor- 
mation, a  state  can  assess  its  situation  and  determine  which 
of  the  methods  would  be  most  appropriate. 
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II.   REPLACEMENT  METHOD  A 

A.  History  and  References 

This  is  the  method  which  has  been  suggested  for  integrating 
Medicaid  QC  sample  selection  with  AFDC  quality  control  and  SSI 
quality  assurance  sample  cases.   This  method  has  been  previously 
used  in  at  least  one  state,  Illinois ,  to  integrate  AFDC  and 
Food  Stamp  QC  sample  selection.   Replacement  Method  A  is  a 
combination  of  these  two  methodologies.   Additional  information 
on  this  method  is  contained  in  the  revised  Medicaid  QC  sampling 
manual  and  An  Evaluation  of  Integrated  Sample  Selection  for 
AFDC  and  Food  Stamp  Quality  Control. 

B.  Sampling  Procedure 

1)  Obtain  the  AFDC  quality  control  sample  in  the  same  way 
this  was  done  with  independent  sample  selection.   In 
addition  to  the  AFDC  quality  control  review,  all  of  these 
cases  will  be  included  in  the  Medicaid  QC  sample.   All  of 
these  cases  which  receive  food  stamps  (N^£)  will  also  be 
included  in  the  food  stamp  QC  sample. 

2)  Obtain  the  remaining  number  of  food  stamp  QC  cases 

(Nf  -  N^^)  from  the  non-AFDC  portion  of  the  food  stamp 
caseload.   N£  is  the  total  number  of  food  stamp  QC  cases 
needed. 

3)  Include  all  SSI  quality  assurance  cases  in  the  Medicaid 
QC  sample. 

4)  Obtain  the  additional  Medicaid  QC   sample  cases  needed 

from  the  portion  of  the  Medicaid  eligibility  file  not 

covered  in  steps  1  or  3.   The  number  needed  is  specified 

for  each  state  in  the  table  of  Medicaid  QC  sample  sizes. 
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C.  Frame  Requirements 

For  this  method  to  be  implemented,  the  following  conditions 

must  be  met. 

1)  It  must  be  possible  to  identify  (at  the  time  of  sample 
selection)  those  AFDC  quality  control  sample  cases  which 
also  receive  food  stamps. 

2)  The  frame  used  to  select  the  food  stamp  QC  sample  cases 
must  specify  which  are  AFDC  cases  and  which  are  non-AFDC 

cases. 

3)  The  frame  used  to  select  Medicaid  cases  must  specify 
which  cases  are  AFDC,  which  cases  are  SSI  and  which  cases 
are  neither  of  these. 

D.  Weighting  Required 

This  method  results  in  the  following  weighting  requirements 

1)  No  weighting  if  required  in  the  compilation  of  AFDC 
quality  control  findings. 

2)  Weighting  is  required  in  the  compilation  of  Medicaid 
and  Food  Stamp  quality  control  findings. 

E.  Relative  Efficiency 

The  exact  amount  of  reduction  in  QC  workloads  which  will 
result  from  integrated  sample  selection  will  vary  from  state 
to  state  due  to  differences  in  the  amount  of  overlap  in  the 
three  caseloads.   The  information  on  relative  efficiency 
which  follows  is  based  on  caseload  data  from  one  state  (West 
Virginia)  from  which  information  on  caseload  overlap  was 
available.   In  West  Virginia,  the  required  QC  sample  sizes 
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are  550  for  AFDC  quality  control,  1200  for  Food  Stamp  quality 
control,  and  925  for  Medicaid  quality  control.   If  no  integrated 
sample  selection  were  used  there  would  be  550  +  1200  +  925  = 
2675  distinct  cases  to  be  reviewed.   Integrating  the  Medicaid 
quality  control  sample  selection  with  AFDC  quality  control  and 
SSI  quality  assurance  results  in  a  decrease  of  550  (or  21^o) 
in  the  number  of  distinct  cases  and  a  reduction  of  about  750 
(or  281)  in  the  number  of  distinct  cases  with  field  investigations, 
since  most  of  the  200  SSI  quality  assurance  cases  would  not 
require  a  field  investigation  for  Medicaid  quality  control. 
Since  about  701  of  the  AFDC  cases  in  West  Virginia  obtain  Food 
Stamps,  Replacement  Method  A  would  produce  about  .70  x  550  =  385 
Food  Stamp  reviews  from  the  AFDC  quality  control  sample.   This 
is  an  additional  141  reduction  in  the  number  of  distinct  cases 
(all  with  field  investigations) .   The  time  saved  in  this  way 
should  more  than  compensate  for  the  additional  time  required 
by  other  aspects  of  the  expanded  Medicaid  QC  sample  (claims 
review  and  third  party  liability  review).   Thus,  with  Replacement 
Method  A,  a  state  should  be  able  to  implement  the  expanded 
Medicaid  QC  system  without  adding  any  additional  QC  staff. 
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III.   REPLACEMENT  METHOD  B 

A.  History  and  References 

This  method  is  very  similar  to  Replacement  Method  A 
except  that  it  produces  self  weighting  samples  for  all  of 
the  QC  systems.   It  is  not  quite  as  efficient  as  Method  A, 
but  if  a  state  is  shorter  on  data  analysis  resources  than 
review  staff,  this  method  would  be  preferable  to  Method  A. 

Replacement  Method  B  has  been  used  in  many  states  to 
integrate  sample  selection  for  AFDC  and  Food  Stamp  Quality 
Control.   It  is  described  in  An  Evaluation  of  Integrated 
Sample  Selection  for  AFDC  and  Food  Stamp  Quality  Control. 

B.  Sampling  Procedure 

1)  Obtain  the  AFDC  quality  control  sample  in  the  same  way 
this  was  done  with  independent  sample  selection. 

2)  Obtain  a  stratified  sample  of  food  stamp  cases  (stratified 
according  to  AFDC/non-AFDC) . 

3)  Replace  all  AFDC  Food  Stamp  cases  in  sample  2  with  the 
same  number  of  AFDC  Food  Stamp  cases  from  sample  1 
(these  must  be  a  random  subsample  of  sample  1) . 

4)  Obtain  a  stratified  sample  of  cases  from  the  Medicaid 
eligibility  file  (stratified  according  to  AFDC,  SSI, 
other) . 

5)  Replace  all  AFDC  cases  in  sample  4  with  a  random  sub- 
sample  of  equal  size  from  sample  1. 

6)  Replace  all  SSI  cases  in  sample  4  with  a  random  sub- 
sample  of  the  same  size  from  the  SSI  quality  assurance 

sample. 
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C.  Frame  Requirements 

The  frame  requirements  for  Replacement  Method  B  are  the 
same  as  for  Replacement  Method  A,  i.e.,  that  in  effect  the 
frames  used  to  select  each  sample  must  contain  some  infor- 
mation about  other  benefits  in  the  case.   This  condition  is 
not  met  in  many  states  and  was  a  barrier  to  integrated 
sample  selection  in  those  states  until  other  methods  were 
developed.   The  methods  described  in  the  following  two  sections 
do  not  require  that  the  benefit  combination  in  cases  be 
indicated  in  the  frames. 

D.  Weighting  Required 

Replacement  Method  B  produces  self  weighting  samples 
for  all  three  QC  systems. 

E.  Relative  Efficiency 

Replacement  Method  B  would  produce  the  following  types 
of  cases  for  QC  review  in  the  example  state. 


Type  of  Case 

AFDC 
Reviews 

Medicaid 
Reviews 

Food  Stamp 
Reviews 

Non-AFDC  Food 
Stamp  Cases 

840 

0 

0 

840 

AFDC  Food 
Stamp  Cases 

385 

385 

199 

360 

Non-Food  Stamp 
AFDC  Cases 

165 

165 

85 

SSI  Cases 

547 

547 

Other  Medicaid 
Cases 

94 

94 

TOTAL  2031  550         925  1200 
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The  2031  cases  to  be  reviewed  is  a  24%  reduction  in  cases 
to  be  reviewed  compared  to  independent  sample  selection.   This 
is  not  as  good  as  the  35%  reduction  with  Replacement  Method  A. 
However,  the  percent  reduction  in  cases  with  field  investigations 
is  greater  with  Method  B  (30%)  than  with  Method  A  (28%). 
Using  independent  sample  selection  there  would  be  1200  +  550  + 
378  =  2128  cases  with  field  investigations.   With  Method  B 
there  are  2031  -  547  =  1484  cases  with  field  investigations. 
With  Method  A  there  are  1540  (56  more)  cases  with  field 
investigations . 
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IV.   SOCIAL  SECURITY  NUMBER  METHOD 

A.   History  and  References 

This  method  was  first  used  by  the  South  Dakota  Department 
of  Social  Services  to  integrate  sample  selection  for  AFDC  and 
Food  Stamp  quality  control.   This  method  is  of  special  interest 
because  it  is  the  only  method  (of  the  five)  which  produces  self 
weighting  samples  without  requiring  that  the  benefit  combination 
of  cases  be  indicated  in  the  sampling  frames. 

B-   Sampling  Procedure 

An  IBM  software  package  called  RANDU  is  used  to  generate 

a  pre-specif led  number  of  4  digit  random  numbers.   The  number 

of  random  numbers  needed  to  select  the  AFDC  quality  control 

sample  may  be  different  from  the  number  of  random  numbers 

needed  to  select  the  Food  Stamp  or  Medicaid  quality  control 

samples.   Let  R^ ,  R^,  and  R  be  the  respective  numbers  of 

aim 

random  numbers  used  for  the  three  samples. 

The  AFDC  quality  control  sample  is  obtained  by  comparing 

the  last  four  digits  of  the  Social  Security  number  of  the  payee 

in  each  AFDC  case  to  all  R   random  numbers  being  used  to  select 

a 

the  AFDC  quality  control  sample.   If  there  is  a  match,  the  case 
is  selected  to  be  included  in  the  AFDC  quality  control  sample. 
The  probability  of  selection  for  any  AFDC  case  is  Rg/lO.OOO. 
R_  is  determined  as  follows. 

a 

^   Average  Monthly  Sample  Size    A  ^  io,ooO 


)nthly  Sample  Size    \  ^  -j^( 
Average  Caseload  Size) 
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If  the  estimated  average  AFDC  caseload  size  is  21,000 
and  the  average  monthly  sample  size  is  550/6  =  92,  then 

R   =    92  X  10,000    ^  44 
^        21,000 

The  Food  Stamp  quality  control  sample  is  selected  in  the 
same  way,  using  the  Social  Security  number  of  the  Food  Stamp 
household  head  and 

Rf  =  Average  Monthly  Sample  Size    ]  ^   ]^q  qoo 
(Estimated  Average  Caseload  Size/      ' 

If  the  estimated  average  food  stamp  caseload  size  is  55,000 
and  the  average  monthly  sample  size  needed  is  200  then 

j^  ^  200  X  10,000   =  37 
^     55,000 

To  obtain  an  integrated  smaple,  37  of  the  same  random  numbers 
used  to  select  the  AFDC  sample  must  be  used. 

The  Medicaid  quality  control  sample  is  selected  in  the  same 
way,  using  the  Social  Security  number  of  the  primary  person  in 
the  Medicaid  case  and 

^  Average  Monthly  Sample  Size     ]  x  10.000 
'^m   I  Estimated  Average  Caseload  Sizey 

If  the  estimated  average  caseload  size  is  71,000  and  the 
average  monthly  sample  size  is  925/6,  then 

R   _  (925/6)x  10,000   ^  ,, 
^       71,000 
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C.  Frame  Requirements 

The  only  frame  requirement   for  this  method  is  that  the 
frames  for  selecting  the  three  quality  control  samples  all 
contain  a  Social  Security  number  for  each  case.   The  statistical 
validity  of  this  method  is  not  affected  by  an  occasional  case 
with  no  Social  Security  number  if  a  provision  is  made  to 
generate  random  numbers  for  the  missing  Social  Security  numbers. 
This  will,  however,  reduce  the  amount  of  overlap  in  the  samples 
and  thus  reduce  the  efficiency  of  the  method. 

D.  Weighting  Required 

The  Social  Security  number  method  produces  self  weighting 
samples  for  all  three  quality  control  systems. 

E.  Relative  Efficiency 

The  Social  Security  number  method  is  somewhat  more  efficient 

than  the  other  method  (Replacement  Method  B)  of  obtaining  self 

weighting  samples.   This  is  because  of  the  167  cases  which 

receive  both  a  Medicaid  and  a  Food  Stamp  review.   The  table 

belov;  shows  the  number  of  cases  of  each  type  in  the  QC  samples 

in  the  example  state. 

Medicaid      Food  Stamp 
Sample  Cases   Sample  Cases 

194  326 

83 

167  284 

491 

610 

554             935  1220 
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Bene 

fit 

Caseload 

AFDC 

Comb 

ination 

Size 

Sample  Cases 

AMF 

14,700 

388 

AM 

6,300 

166 

MF 

12,800 

MO 

37,200 

FO 

27,500 

The  total  number  of  distinct  cases  is  1928,  whicli  is  28"; 
less  than  the  total  number  of  reviews  (2709).   In  some  states, 
including  South  Dakota,  the  relative  efficiency  of  this  method 
would  be  much  higher  due  to  more  similarity  in  the  number  of 
random  numbers  needed  to  select  the  three  samples. 
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V.   SEQUENTIAL  RANDOM  SAMPLE  METHOD 

A.  History  and  References 

This  method  was  first  used  in  West  Virginia  to  integrate 
the  sample  selection  for  AFDC,  Food  Stamp,  and  Medicaid  quality 
control.   It  is  more  efficient  than  any  of  the  previous  methods 
and  it  does  not  require  stratified  frames  (i.e.  that  the  benefit 
combination  be  known  at  the  time  of  QC  sample  selection) .   Its 
main  disadvantage  is  that  weighting  must  always  be  used  in  the 
compilation  of  quality  control  findings.   This  method  is  described 
in  West  Virginia  Department  of  Welfare  Integrated  Sample  Selection 
for  AFDC,  Food  Stamp  and  Medicaid  Quality  Control. 

B.  Sampling  Procedure 

1)  Obtain  a  sequential  random  sample  of  cases  from  the  complete 
AFDC  frame,  using  an  interval  of  I^. 

2)  Obtain  a  sequential  random  sample  of  cases  from  the  complete 
Food  Stamp  frame,  using  an  interval  of  l£. 

3)  Obtain  a  sequential  random  sample  of  cases  crom  the  complete 
Medicaid  eligibility  file,  using  an  interval  of  I^. 

4)  In  each  of  the  cases  selected,  review  all  of  the  benefits 
(AFDC,  Food  Stamps,  and  Medicaid)  that  are  received  in 
the  review  month. 

C.  Frame  Requirements 

This  is  the  only  method  which  does  not  have  any  frame 
requirements  in  additional  to  the  frames  that  would  be  required 
to  select  independent  samples  for  each  of  the  quality  control 
systems . 
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D.  Weighting  Required 

Cases  with  different  benefit  combinations  will  have 
different  probabilities  of  being  included  in  the  integrated 
QC  sample.   When  tabulating  QC  findings  it  will  be  necessary 
to  weight  each  case  according  to  the  inverse  of  its  probability 
of  selection.   These  weights  are  a  simple  function  of  the 
effective  sampling  intervals  as  described  in  the  reference. 

E.  Relative  Efficiency 

This  method  is  considerably  more  efficient  than  any  of 
the  previous  methods  because  multiple  benefit  cases  have  a 
higher  probability  of  being  included  in  the  sample  and  all  of 
the  benefits  in  each  sample  case  are  reviewed  (this  is  not  true 
in  any  of  the  previous  methods). 

The  table  below  shows  the  approximate  sample  sizes  which 
would  be  obtained  in  the  example  state. 


CASELOAD 
SIZE 

SAMPLING 
INTERVAL 

504 

SAMPLE 
SIZE 

250 

AFDC 
REVIEWS 

250 

FOOD  STAMP 
REVIEWS 

175 

MEDICAIE 
REVIEWS 

AFDC 

21,000 

250 

Food 
Stamps 

55,000 

298 

1107 

354 

1107 

553 

Medicaid 

71,000 

2477 

172 

46 

68 

172 

TOTAL 

1529 

650 

1350 

975 

The  total 

number  of 

reviews  is 

650  +  1350 

+  975  =  2975 

.   The 

number  of  cases  to  be  reviewed  is  1529,  which  is  a  49%  reduction 
in  the  number  of  cases  that  would  have  to  be  reviewed  if 
independent  sample  selection  were  used. 
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VI.   OPTIMUM  ALLOCATION  METHODS 

A.  History  and  References 

The  previous  methods  all  result  in  the  selection  of  different 
numbers  of  cases  with  each  benefit  combination.   None  of  the 
allocations  are  optimal  in  the  sense  of  minimizing  the  variance 
of  error  rate  estimates.   The  optimal  number  of  cases  with  a 
particular  benefit  combination  to  be  selected  can  be  computed 
once  it  is  decided  what  to  minimize  and  what  the  constraints 
are.   This  is  an  area  where  some  policy  decisions  need  to  be 
made.   Should  there  be  constraints  on  the  number  of  cases  reviewed 
or  on  the  variance  of  estimates?   Is  it  necessary  that  the 
required  QC  sample  sizes  be  step  functions?   Integrated  sampling 
could  be  much  easier  and  the  need  for  weighting  could  be  elimi- 
nated by  using  the  same  sampling  fraction  or  interval  in  each 
QC  system. 

An  example  of  an  optimum  allocation  of  cases  based  on  one 
set  of  assumptions  has  been  worked  out  and  described  by 
Manny  Schwartz. 

B.  Sampling  Procedure 

For  an  optimal  allocation  design,  each  case  would  ideally 
be  exposed  to  sampling  only  once  and  would  be  selected  with 
a  probability  which  would  be  the  same  for  all  cases  with  the 
same  benefit  combination  and  different  for  cases  with  different 
benefit  combinations. 
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C.  Frame  Requirements 

To  select  cases  in  the  most  optimal  manner,  it  would  be 
necessary  to  have  a  single  frame  {computer  file  or  listing 
of  the  combined  AFDC,  Food  Stamp  and  Medicaid  caseloads). 
Each  case  should  be  listed  only  once  and  tlie  benefit  combination 
should  be  indicated.   Unless  a  computer  file  of  this  type 
already  exists,  it  would  probably  not  be  worth  the  effort  to 
create  such  a  frame  just  for  the  purpose  of  integrated  sample 
selection. 

Near  optimal  allocations  could  be  obtained  without  a 
merged  frame  if  the  separate  frames  indicate  what  other  benefits 
each  case  is  receiving. 

D.  Weighting  Requirements 

Any  optimated  sample  will  require  that  the  sample  cases 
be  weighted  according  to  the  inverse  of  the  probability  of 
selection  for  that  type  of  case. 

E.  Relative  Efficiency 

The  measure  of  efficiency  that  has  been  used  here  to 
compare  the  various  methods  of  obtaining  integrated  samples 
is  the  numl^er  of  cases  reviewed  divided  by  the  number  of  reviews 
obtained.   It  is  unlikely  that  any  optimum  allocation  will  be  more 
efficient  in  this  respect  than  the  sequential  random  sample 
method,  which  selects  more  multiple  benefit  cases  than  one 
would  go  after  if  trying  to  minimize  the  variance  of  error 
rate  est  imates . 
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VTII.   SUMMARY 

The  chart  on  the  following  page  is  a  siuiimary  of  some  of  the 
main  differences  among  the  various  methods  of  obtaining  inte- 
grated samples  for  AFDC ,  Medicaid,  and  Food  Stamp  Quality 
Control.   The  method  with  the  most  advantages  is  the  Social 
Security  number  method,  which  does  not  require  stratified 
frames  and  produces  self  weighting  samples,   This  is  not  the 
most  efficient  method,  but  it  is  the  most  efficient  method  of 
obtaining  self  weighting  samples.   The  more  efficient  methods 
all  require  weighting  of  the  data  and,  except  for  the  sequential 
random  sample  method,  they  all  require  stratified  frames 
which  might  not  exist  in  some  states. 

The  process  of  selecting  an  integrated  sampling  method 
would  involve,  first  of  all,  eliminating  those  which  could  not 
be  implemented  because  of  some  barrier  such  as  the  lack  of 
required  frames.   Then,  if  more  than  one  method  could  be 
iraiplemented,  priorities  must  be  determined  for  the  other  features 
For  example,  which  is  more  important,  more  efficiency  or  self 
weighting  samples? 

Since  one  of  the  methods  has  no  frame  requirements  beyond 
what  is  needed  to  select  independent  samples,  it  is  safe  to 
say  that  at  least  one  of  tlie  methods  could  be  implemented  in 
every  state.   There  is  enough  efficiency  in  even  the  least 
efficient  methods  to  compensate  for  the  extra  effort  that  will 
be  involved  in  the  expanded  Medicaid  Quality  Control  system. 
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FRAME 
REQUIREMENTS 


WEIGHTING 
REQUIRED? 


PERCENT  REDUCTION 
IN  CASES  REVIEWED 


1  .  Replacement 
Method  A 


Stratified 
Frames 


Yes-except 
for  AFDC 


5Sl 


2.    Replacement 
Method  B 


Stratified 

Frames 


No 


24' 


Social  Security 
Number  Method 


Social  Security 
No.  in  Frames 


No 


28% 


4.  Sequential 
Random  Sample 


None 


Yes 


491 


5.  Optimum 
Allocation 
Methods 


Integrated 
Frame 


Yes 
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DESIGNING  OPTIMAL  INTEGRATED  SAMPLES 


By:   Rudolph  E.  Schwartz 

Office  of  Research  and  Statistics 
Social  Security  Administration 
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Designing  Optimal  Integrated  Samples 
Rudolph  E,  Schwartz,  Office  of  Research  and  Statistics 
Social  Security  Administration 


At  present,  there  are  in  existence  at  least  three  q\iality  control 

programs  which  the  States  are  required  to  carry  out  imder  Federal 

guidelines.  They  are: 

Aid  for  Dependent  Children  -  APDC 
Food  Stamps  -  FS 
Medicaid  -  Md 

The  purpose  of  each  of  the  quality  control  programs  is  to  measiire  and 
by  corrective  action  reduce  the  frequency  of  errors  which  are  either 
payments  to  ineligible  recipients  or  over  and  under  payments  to  eligibles. 

The  three  programs  have  many  similarities.  Each  consists  of  monthly 
simple  random  samples  of  beneficiaries.  Each  program  is  operated 
independently  with  its  own  program  specifications  and,  in  many  States, 
separate  reviewers  for  each  of  the  various  programs. 

Lately,  under  the  pressure  of  increased  cost  and  sample  size  demands, 
the  States  have  begun  to  review  the  assumptions  of  running  each  program 
separately.   It  has  been  known  that  in  many  instances  there  is  substantial 
overlap  between  recipients  and  programs.  For  example,  an  AFDC  recipient 
mi^t  be  eligible  for  both  Food  Stamps  and  Medicaid,  or  a  Food  Stamp 
recipient  might  receive  Medicaid,  etc. 
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Therefore,  many  States  have  tegun  to  see  the  advantage  of  thus  interviewing 
for  one  program  and  receiving  information  that  can  he  used  by  several  programs. 

In  some  States,  there  has  been  an   effort  to  cross-train  people  trained  in 
interviewing  for  several  programs.  People  thus  trained  are  called  generic 
reviewers  as  opposed  to  specialized  single  program  reviewers. 

In  a  few  States  that  are  making  efforts  at  integration,  it  has  been  shown 
that  it  is  possible  to  achieve  savings  of  between  30-1+0  percent  by  integrating 
as  opposed  to  running  separate  programs. 

Given  that  one  has  generic  reviewers  the  sample  design  can  be  done  in  various 
viay  s . 

1 .  One  can  continue  to  choose  independent  samples  from  each  of  the 
program  files  and  after  selection  merge  cases  on  individuals 
that  belong  to  more  than  one  program. 

2.  One  can  merge  all  three  program  files  and  constmict  mutually 
exclusive  strata  consisting  of  a  maximum  of  seven  (?)  strata 
as  follows: 


(1)  AFDC  only;  (2)  APDC-FS;  (3)  APDC-FS-Md;  (U)  APDC-Md; 
(5)  FS  only;  (6)  FS-Md;  (7)  Md  only. 
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Having  done  this  the  problem  became  one  of  designing  a  sample  that  is 
optimal  for  all  three  programs  combined.  This  paper  describes  the  second 
approach. 
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An  optimal  sample  takes  into  consideration  3  elements: 

(1)  Reliability  or  allowable  sampling  error. 

(2)  Variability  of  the  universe  or  sub-imiverses  from  which  sajmples 
are  drawn. 

(3)  Cost  of  sampling  which  considers  setup,  travel,  interview  and 
verification  costs. 

The  difficulty  with  designing  an  optimal  sample  is  that  each  of  the  various 
programs  has  its  own  values  for  the  three  elements  mentioned  above.  Thus 
the  sample  that  is  optimal  for  one  program  may  not  be  optimal  for  another. 

In  order  to  solve  such  a  problem  for  stratified  samples  and  several  programs 
one  must  solve  a  nonlinear  programming  problem. 

Mathematical  Formulation  of  an  Optimal  Stratified  Integrated  Sampling 
Problem  Samples 

MIN  C  =  Z  a^n^  J  =  1  •  •  -J  programs 

Subject  to  h  =  1  .  .  .  L  strata 

L 


V.  £  C.   (neglecting  finite  correction  factor)  Y.  =1^-1  Wfl 
J     J  J   n— X  n 


W?S 


M 


% 

where  N   =  Total  units  of  population 

N,   =  Total  units  in  h  stratum 
h 

n.   =  Sample  units  in  hth  stratum 
V   =  N, /N  =  Stratum  weight 

2 

S,   =  estimate  of  population  variance  in  h  stratum 
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Below  is  an  example  of  a  typical  optimal  solution  which  has  been  solved 
by  a  nonlinear  programming  package. 

Example  -  Design  of  an  optimal  integrated  sample.  Assume  the  present  three 
programs  have  the  following  sample  size  requirements. 

Program Sample    SRS  Variance*  (lO"^) 

Aid  for  Dependent  Children  (AFDC)       800       1.10 
Food  Stamps  (PS)  1,200       0.61 

Medicaid  (MED)  275       2.76 

*SRS  Variance  =  Simple  random  sample  variance  equivalent  for  sample  sizes 
if  P  =  .095 

Construct  mutually  exclusive  strata  by  merging  the  files  of  the  three 
programs  into  seven  mutually  exclusive  strata. 


Stratum 

Popxilation 

Cost 

Unit 

No. 

Size 

.  Factor 

Sampling 

Cost 

(000) 

1 

AFDC 

9 

1 

9.50 

9.50 

2 

APDC,   PS 

18 

1/2 

11.50 

5.75 

3 

APDC,   PS,  MED 

2 

V^ 

13.50 

i;.50 

k 

APDC,   MED 

1 

1/2 

11.50 

5.75 

5 

PS 

23 

1 

9.50 

9.50 

6 

PS,  MED 

7 

1/2 

11.50 

5.75 

7 

MED 

hS 

1 

9.50 

9.50 

MINIMIZE: 

C  =  9.50n.   +  11.50n2  +  13.50n^  +  11.50n,    +  9.50nK  +  11.50ng  +  9.50n^ 

SUBJECT  TO: 

0  1  n^    1 N^    .    .    ;   0  1  n     <  N  „ 


^30^^  n^  ^30^^  ng  ^30^  n^  ^30^  n,  -x10-i+ 

,18  2  (^22)^^-^22)  ,  ,^2  (P3,)(1-P3,)  ^  ^2  (Pg,)(l-P^,)  ^      .2  (P62)(l-^62)  <o.6l 
^^50^  n^  ^W  n^  ^W  n^  ^^0^  n^  -x10-l+ 

2  (P33)(1-P33)  ^       .2   (P^3)(1-P^3)  ^       .2  (P63)(1-P63)  ,     ,^2  (P,3)(1-P,3)  ,2.76 
^S9  n^  ''SS^  n^  '^55^  ng  ^55^  n^  "xlO-U 

Where  P^^   =  09,  ^^^   =  .10,  P3^   =  08,   P^^   =  07 

P22  =  06,  P^2  =  09,  P52  =  •''°'  ^62  =  °^ 
P^^  =  08,  P^^  =  .10,  Pg3  =  07,  P^3  =  .10 

n^  =  205,  n2  =  50I1,  n^  =  5U,  n,    =  19,  n^  =  S6h,  n^  =  1M+,  n^  =  226 

Cost  =  17852  Hrs. 
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Table  1  shows  the  results  of  constrasting  a  proportional  sample  which  assumes 
that  sampling  costs  and  variability  in  all  stratum  are  approximately  equal 
and  an  optimal  sample.  The  table  is  read  as  follows: 

Column 

1  Program  name  and  strata  used  by  the  program 

2  Population  size  of  each  stratum  u.sed  by  each  program 

3  Wei^t  =  N.A^^. 

h  P  =  estimate  of  proportion  of  error  in  each  stratum  for  the 

various  programs 

5  Theoretical  proportional  sample  allocation  (this  sample  would  be 

self -weighting) 

6  Actual  proportional  sample  used 

Example  -  FS  stratum  3  requires  only  i+S  cases,  however,  since  Sh 
cases  have  been  finished  to  satisfy  the  requirements 
of  APDC  stratum  3  it  would  not  be  sensible  to  throw 
away  the  I4.  cases  already  completed  unless  self- 
weighting  is  crucial. 

7  Optimal  sample  allocation 

Referring  to  table  1  we  can  see  that  optimal  sample  when  compared  with  the 
proportional  sample  is  cheaper  in  the  APDC  and  MED  strata  but  sli^tly  more 
expensive  in  the  FS  strata. 

However,  compared  to  independent  simple  random  saples,  both  AFDC  and  FS  are 
substantially  cheaper  while  Medicaid  is  slightly  more  expensive. 
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Table  I  -  Contrast  Between  Proportional  Sample  and  Optimal  Sample 


(1) 

(2) 

(3) 

(h) 

(5) 

Theor. 

(6) 

Actiial 

(7) 

Pop. 

WT 

P 

Prop. 

Prop. 

Optimal 

(000) 

ATOC 

1 

9 

.300 

.09 

239 

239 

201+ 

2 

18 

600 

10 

1+80 

i+80 

1+83 

3 

2 

067 

08 

51; 

51+ 

1+2 

h 

1 

033 

07 

27 

27 

18 

Total 

30 

800 

800 

757 

Var(lO~^) 

1.10 

1.10 

1.10 

Cost  - 

5,1+29 

5,065 

-SES* 

7,600 

FS 

2 

18 

.36 

.06 

1+32' 

i+80 

1+93 

3 

2 

01+ 

09 

1+8 

51+- 

1+2 

5 

23 

h6 

10 

552 

552 

578 

6 

7 

11+ 

08 

168 

168 

11+7 

Total 

50 

1,200 

1,251+ 

1,260 

Var(lO"^) 

0.63 

0.61 

0.61 

Cost  - 

9,213 

9,860 

-SRS 

11,1+00 

mi\) 

3 

2 

.036 

.08 

10 

51+ 

1+2 

U 

1 

018 

10 

5 

27 

18 

6 

7 

127 

07 

35 

168 

11+7 

7 

ii5 

819 

10 

225 

225 

200 

Total 

55 

275 

l+7i+ 

1+07 

Var(lO~^) 

3.12 

2.76 

3.12 

Cost  - 

3,502 

3,037 

-SRS 

2,613 

Size   (1)  Net 

Size 

1,505 

1,559 

1,716 

(2)  Gross  Size 

2,528 

2,1+91 

*SRS  -  Simpl< 

5  Random 

Sample 

Cost. 

stratum 
No. 

Program  Name 

1 

AFDC 

2 

AFDC,  PS 

3 

APDC,  FS,  MED 

k 

AFDC,   MED 

5 

Jj-S 

6 

PS,  MED 

7 

MED 
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CRITERIA  FOR  SUBMISSION  OF  INTEGRATED 
SAMPLE  DESIGN  PROPOSALS 


By:   Sean  D.  Hurley,  Acting  Chief 
Technical  Services  Branch 
Division  of  Quality  Control 

Management 
Social  Security  Administration 
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Criteria  for  Submission  of  Integrated  QC  Sample  Design  Proposals 

Integration  of  one  or  more  of  the  Quality  Control  systems  involves  a 
variety  of  statistical,  policy,  and  procedural  problems.   The 
Department  of  Health,  Education,  and  Welfare  believes  that  each  State 
wishing  to  develop  an  integrated  sample  design  is  best  qualified  to 
search  out  solutions  within  the  frame  work  of  its  own  system.   Each 
interested  State  should  submit  its  proposal  for  integration  concurrently 
to  the  respective  Regional  Offices  (SSA/DA/HCFA)  for  review.   The 
Regional  Offices,  in  turn,  will  evaluate  the  proposal  and  provide 
technical  assistance  to  the  State,  where  needed,  to  insure  adherence  to 
the  necessary  criteria.   The  Regional  Offices  must  then  submit  the 
proposal,  with  recommendations,  to  their  respective  Central  Offices  for 
final  approval.   (This  procedure  will  facilitate  the  close  coordination 
required  between  the  various  Agencies  and  insure  that  States  are  not 
operating  under  an  unapprovable  plan.) 

State  proposals  for  integration  will  be  reviewed  and  approved  on  a  State 
by  State  basis.   Although  precise  criteria  for  the  integration  of  all 
three  programs  has  not  been  established.  States  submitting  proposals 
should  address  the  following  issues: 

A.   Integrity  of  Sample 

Design 

1.  Describe  the  method  to  be  used  in  sample  integration. 

2.  Describe  how  the  resulting  sample  will  insure  the  validity  of 
sample  findings  for  each  of  the  programs  integrated. 
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3.   Describe  what  impact  the  sample  design  will  have  on  reporting 
and  data  requirements. 

B.  Integrity  of  the  State  Case  Review  Process 
Review  Process 

1.  Describe  what  impact  the  sample  design  will  have  on  the  review 
process.  Consider  that  each  program  may  have  different  review 
criteria,  time  constraints,  and  procedures. 

2.  Describe  the  case  review  process  under  the  integrated  sample 
design.   Propose  resolutions  to  problems  created  by  integration 
of  the  review  process. 

3.  Describe  plans  for  staff  training  to  insure  the  quality  of  case 
reviews.   Describe  the  mechanism  to  be  used  to  measure  any  change 
in  the  quality  of  reviews. 

C .  Corrective  Action  Process 

Describe  what  changes,  if  any,  will  occur  in  the  corrective  action 
process . 

D.  Allocation  of  Staff 

1.   Describe  what  impact  the  integrated  sample  design  will  have  on 

reviewer's  workloads.   What  will  be  the  average  number  of  reviews 
completed  per  month  per  reviewer  for  each  type  of  case  review 
(integrated  vs  separate  review,  etc.). 
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2.  Provide  assurance  that  each  of  the  samples  will  be  completed 
within  the  time  frames  required. 

3.  Show  proposed  allocation  of  existing  AFDC ,  FS ,  and  MEQC  staff. 

4.  Indicate  what,  if  any,  additional  staff  are  required. 

E .  Cost  Allocation 

Show  how  QC  administrative  costs  will  be  apportioned  correctly 
between  USDA  and  HEW. 

F.  Administration 

Describe  changes,  if  any,  in  organization  and  management  of  integrated 
QC  process . 

In  addition  to  the  above  considerations.  State  estimates  of  the  potential 
cost  savings  would  be  helpful.   States  planning  to  integrate  their  QC 
systems  should  submit  their  proposals  as  soon  as  possible  and  for  their 
own  protection,  should  not  implement  their  plans  until  Central  Office 
approval  has  been  obtained. 
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Section  III  -  State  Presentations 

Several  States  with  experience  in  integrated  Quality 
Control  samples  were  asked  to  make  presentations 
focused  on  highlighting  specific  aspects  of  their 
systems.   Illinois  and  Rhode  Island,  for  example 
outlined  the  impact  sample  integration  had  upon  the 
review  process.   Colorado  and  New  York  explained 
their  respective  sampling  designs,  while  Montana 
discussed  the  cost  benefits  realized  as  a  result  of 
their  integrated  sample  design. 
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MONTANA  -  COST  EFFECTIVENESS 


Presented  by:   Ann  Murray,  Chief 
Quality  Control 
Montana  Department  of  Social 
and  Rehabilitation  Services 
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In  1976,  Montana  Quality  Control  found  it  impossible  to  complete  the 
required  number  of  reviews  with  three  Food  Stamp  Reviewers  and  four 
A.D.C.  Reviewers.   It  was  decided  that  the  only  alternative  was  inte- 
gration of  the  two  programs. 

This  was  accomplished  July  1,  1976.   In  the  first  six  months  we  saved 
15,000  miles  travel,  broken  down  at  55  miles  an  hour  saved  about  38 
working  days  and  we  were  able  to  get  the  job  done. 

July  1,  1977,  our  Medical  Quality  Control  1115  Project  ended.   The 
legislature  only  allowed  retention  of  three  of  the  staff  for  reviewers 
and  the  program  was  placed  under  the  Quality  Control  Supervisor.   At 
that  time  we  decided  to  integrate  staff  and  add  the  Medical  Quality 
Control  to  the  already  integrated  A.D.C.  Food  Stamp  Program. 

We  are  unable  to  make  comparisons  as  to  travel  time  saved  by  doing  this 
because  of  difference  in  size  of  staff  but  we  have  worked  out  a  comparison 
on  miles  traveled  and  travel  costs  per  review. 

Miles  Cost  Meals  Lodging  Total  #Reviews 

1/1/76-6/30/76   /18,000  $7,200  $2,400  $1,900  $11,500  7 

7/1/76-12/31/76  33,000     4,950  1,800  1,400  8,150  7 

1/1/77-6/30/77   35,582     5,337  1,000  737  7,074  6^ 

7/1/77-12/31/77  40,000     6,000  1,194  820  8,014  10 

Travel  and  per  diem  cost  per  review 

#Cases    Miles     Reviews    Miles  per  Cost  per 
Completed  Per  Case  Completed  Review    Review 

1/1/76-6/30/76   400 

7/1/76-12/31/76  350 

1/1/77-6/30/77   404 

7/1/77-12/31/77  505 
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400 
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425 

86 

484 

79 

571 
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70 

10.51 

STATE  OF  MONTANA 
SOCIAL  AND  REHABILITATION  SERVICES 


Ann  Murray  W  ^  ^^,^   May  18,  1977 

Quality  Control  Supervisor 

Jack  Carlson,  Administrator 
Economic  Assistance  Division 

Savings---  '  Integrated  Sampling  Plan 

I  would  like  to  share  with  you  the  following  report  I  received 

from  Statistics",    This  is  a  comparison  of  expenses  involved 

January  thru  June,  1975  before  integration  and  July  thru  Decem- 
ber 1976,  after  integration, 

TRAVEL:   Jan.  thru  June  19  76       48,000  miles 

July  thru  December       33,000  miles   saved  15,000  miles  (31 

MEALS:    Jan  thru  June  $2,400,00 

July  thru  Dec,  1,800,00      (25%) 

LODGING:  Jan  thru  June  $1,900.00 

July  thru  Dec,  1,400.00      (26%)         ; 

Jan  thru  June--each  completed  review     120  miles 
July  thru  Dec.    "        "        "        9  5 

"         "       '"  "  "  7  8      ■"  ..^yM^,^.^uCi^ 

15,000  miles  averaging  50  miles  an  hour  saved  38  days    $1,383,20 

15000  miles  at  15<^  a  miles  2,250.00 

Saving  on  Meals  6  0  0,00 

"    "   Lodging  \-   500.00 

Total  savings  6  months  $4,733.20 

If  this  pattern  holds ,' we  should  save  $20,000  a  year  after 
integration  of  the  Medical  Quality  Control  unit.    This  has  been  ' 
accomplished  only  with  the  hard  work  and  dedication  of  the  entire 
unit. 

This  also  involves  more  completed  cases  than  were  being  done  in 
the  January  thru  June  period--(475  to   540), 
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RHODE  ISLAND  -  IMPACT  ON  REVIEW  PROCESS 


Presented  by:   Frederick  Hajjar,  Administrator 
Standards  and  Planning 
Rhode  Island  Department  of  Social 
and  Rehabilitative  Services 
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In  the  good  old  days,  before  the  quality  control  units  were  merged,  there 
were  two  quality  control  units.   The  AFDC  QC  Unit  consisted  of  one 
supervisor  and  five  reviewers  and  the  Food  Stamp  QC  Unit  consisted  of  one 
supervisor  and  five  reviewers.   As  can  be  seen,  the  samples  were  completed 
in  both  instances,  with  a  case  per  reviewer  per  month  rate  of  10.2.   (1st 
overhead) 

On  July  1,  1976,  when  the  universe  was  expanded  to  include  Food  Stamp 
recipients  who  also  were  receiving  assistance  payments,  it  can  be  seen 
that,  while  the  AFDC  sample  was  completed,  the  Food  Stamp  sample  completion 
rate  was  still  at  the  level  of  the  prior  6-month  period.   (2nd  overhead) 

The  units  were  merged  during  this  6-month  period  in  order  to  maximize  the 
review  of  the  cases  drawn  relative  to  the  number  of  reviewers  on  hand  to 
complete  the  cases.   However,  this  merging  of  the  quality  control  units 
did  not  begin  to  have  an  effect  on  the  completion  rate  until  the  January- 
June,  1977  sample  period.   As  can  be  seen,  the  AFDC  sample  was  again 
completed,  but  while  the  Food  Stamp  sample  was  not  completed,  you  can  see 
that  the  completion  rate  was  increased  from  38%  for  the  initial  6-month 
period  (July  to  December,  1976)  to  69%  for  the  second  6-month  period 
(January  to  June,  1977).   Also,  it  can  be  seen  that  the  productivity  of 
the  reviewers  increased  from  9.9  cases  per  reviewer  per  month  to  15.2 
cases  per  reviewer  per  month.   (3rd  overhead) 

The  following  6-month  period,  that  is,  the  July-December,  1977  period, 
shows  an  increase  in  the  completion  rate  to  79%  compared  to  the  69%  of 
the  previous  6  months.   The  productivity  of  the  reviewers,  however,  leveled 
off  to  13.9  cases  per  reviewer  per  month  as  opposed  to  the  15.2  cases  per 
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reviewer  per  month  for  the  previous  six-month  period.   This  can  be 
attributed  to  the  low  productivity  of  the  new  staff  that  we  added  to 
aid  in  completing  the  larger  Food  Stamp  sample.   (4th  overhead) 

Also,  as  can  be  seen,  the  ineligibility  rate  has  not  only  been  kept  at  a 
very  low  level,  i.e.,  3.6%  AFDC  prior  to  the  merger  and  3.2%  for  the 
later  six-month  period,  but  was  reduced.   Also,  Food  Stamp  ineligibility 
was  decreased  from  23.9%  for  the  period  prior  to  the  merger  of  the  units 
to  3.4%  for  the  latest  six-month  period.   The  overall  error  rate  for 
Food  Stamps  has  decreased  from  45.1%  prior  to  the  merger  to  24.6%  for  the 
latest  six-month  period. 
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PRE-FIERGING  OF  QllAI  ITY  CONTROL  UNITS 


1/1/75  TO  6/30/75 


AFI1£ 


FOOIl  STAMPS 


30 


MAN-MONTHS 


30 


IB 


CASES   DRAWN 


349 


308 


CASES   COMPLETED 


304 


10.3 


cases/reviewer/month 


10.1  combined 
10.2 


ERROR  rate: 


3.57o  (11) 


INELIGIBLE 


23.9%  (62) 


9.7%  (30) 


overpaid 

overcharged/issued 

undercharged/issued 


21.2%  (14) (4) 
(35) (2) 


11  +  30  =  41   number  of  error  cases     62+14+35+4+2  =  117 
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2ND  6  flQNTHS  OF  MERGED  UNITS 


1/1/77  TO  6/30/77 


OL 


33 


MAN-MONTHS 


FOOD  STAMPS 
28 


329 


CASES  DRAWN 


835 


3r5 


9.5 


CASES  COMPLETED 


cases/reviewer/month 


610  - 


k   COMPLETION 
RATE 


21.8  -  COMBINED  15.2 


ERROR  RATE: 


3.95%  (13)       INELIGIBLE 


9.3%  (57) 


7.6%  (25) 


overpaid 

overcharge/issued 

undercharge/issued 


23.1  (37)  (3) 
(100)  (1) 


38 


number  of  error  cases 


1^1 
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5RD  6  MTHS  OF  MFRGFfl  HNTT.<^ 


7/1/77  TO  12/31/77 


Am 


FOOD  STAMPS 


36 


MAN-MONTHS 


36 


322 


CASES  DRAWN 


884 


309 


CASES  COMPLETED 


695  -  79%  COMPLE- 
TION RATE 


8.6 


cases/reviewer/month 


19,3  -  COMBINED 

13.9 


error  rate: 


3.2%  (9) 


INELIGIBLE 


3. 


5.8%  (16) 


overpaid 

overcharge/issued 

undercharge/issued 


21. 2Z  (34)  (0) 
(71)  (1) 


25 


number  of  error  cases 


106 


9^1-^ 


1ST  6  MONTHS  OF  MERGED  UNITS 


7/1/76  TO  12/31/76 


AFIK 


FOOD  STAMPS 


3^ 


MAN-MONTHS 


28 


321 


307 


CASES  DRAWN 


CASES  COMPLETED 


553(SHOULD  HAVE 
DRAWN  800- 
OVERSAMPLE 

875) 
30^  -  38%  coMP. 

RATE 


9.0 


cases/revi ewer/month 


10.8  -  COMBINED  = 

9.9 


ERROR  rate: 


2.6%  (8) 


INELIGIBLE 


8.1%  (30) 


8.1%  (25) 


overpaid 

overcharged/issued 

undercharged/issued 


20%  (20)  (3) 
(51)  (0) 


33 


number  of  error  cases 


104 
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ILLINOIS  -  IMPACT  ON  REVIEW  PROCESS 


Presented  By:   Eugene  Muffler,  Assistant  Chief 
Bureau  of  Quality  Control 
Illinois  Department  of  Public  Aid 
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IMPACT  OF  THE  INTEGRATED  AFDC-FS  SAMPLE  ON  THE  QC  REVIEW  PROCESS 

When  USDA-FNS  required  that  AFDC  cases  be  included  in  the  Food  Stamp  QC 
Review,  Illinois  was  quick  to  develop  interest  in  an  integrated  sample. 
The  reasons  for  this  interest  included: 

1.  The  fact  we  have  already  integrated  Review  Staff.   We  had  changed 
from  the  specialist  to  the  generic  approach. 

2.  It  appeared  to  be  a  way  of  getting  two  reviews  for  the  price  of  one. 

3.  Local  offices  had  complained  of  the  number  of  various  people  who  re- 
view cases  and  this  seemed  like  a  way  to  reduce  this  outside  traffic 
in  the  local  office. 

4.  It  seemed  like  a  sound  idea  to  gather  information  which  would  show 
the  inter-relationships  in  different  programs  for  the  same  case. 

As  always  nothing  is  quite  as  simple  or  as  productive  as  it  first  sounds, 
some  problems  are  bound  to  appear. 

It  seems  appropriate,  at  this  point,  to  describe,  briefly,  our  sampling 
method  and  to  indicate  a  couple  of  problem  areas.   The  existing  AFDC 
sample  was  utilized  with  no  changes  to  obtain  the  sample  for  the  PA-FS 
Review.   This  involved  1200  cases  plus  an  over  sample.   It  was  known  all 
PA  cases  were  not  certified  to  participate  in  the  Food  Stamp  program, 
but  it  could  be  projected  that  870  completed  PA-FS  Reviews  would  be  pos- 
sible.  An  additional  sample  of  330  NA-FS  cases  was  then  pulled  to  bring 
the  total  FS  samples  to  the  required  1200  case  level.   To  complete  both 
the  AFDC  and  FS  Reviews  a  total  of  2400  reviews  were  required.   This  sam- 
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pling  plan  would  provide  the  2400  reviews  but  would  require  that  only  1530 
cases  be  reviewed: 

330  pure  AFDC 
870  AFDC-FS  cases 
330  NA-FS  cases 

Two  major  problems  for  the  review  process  resulted  from  this  sampling 
method.   First,  since  the  PA-FS  sample  was  pulled  from  the  AFDC  Authoriza- 
tion file  those  cases  certified  to  receive  FS  had  to  be  identified.   By 
our  automated  files,  a  check  could  be  made  and  on  the  sample  list  itself 
data  on  FS  certification  was  provided  for  most  cases.   Others,  such  as 
Emergency  Authorization  and  some  initial  approvals,  could  not  be  counted 
as  certified  until  the  case  record  was  audited.   Those  that  were  not  cert- 
ified for  FS  were  dropped  as  "listed  in  error".   In  effect,  this  meant  we 
had  to  look  at  330  cases  in  some  manner  that  would  not  be  reviewed  for  FS. 
This  really  created  no  difficulty  once  the  initial  adjustment  was  made  and 
staff  began  to  remember  to  do  this  in  the  initial  audit.   The  time  required 
for  this  check  is  minimal  in  most  cases.   Secondly,  for  both  PA  and  NA-FS 
Reviews,  the  QC  Review  had  to  determine  participation.   Due  to  internal  pro- 
blems, we  have  never  been  able  to  draw  a  FS  sample  from  a  participation  file. 
In  order  to  secure  a  timely  sample,  we  had  to  rely  on  the  certification  file 
for  sample  selection.   In  both  the  PA  and  NA  reviews,  the  first  step  was  to 
determine  if  the  household  purchased  stamps  in  the  review  month.   This  is 
accomplished  during  the  client  interview. 

Drawing  the  FS  sample  in  this  manner  created  some  reporting  problems.   Our 
Research  and  Statistics  Unit  developed  a  method  of  weighting  PA  and  NA 
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cases  for  reporting  findings  so  that  a  combined  report  could  be  made. 

A  very  large  factor  to  be  considered  in  the  integrated  sample  approach  is 
the  staff  training  or  retraining  which  is  required  when  a  specialist  con- 
cept has  been  utilized.   Illinois  had  integrated  review  staff  several 
years  earlier.   This  change  had  been  dictated  by  a  need  to  have  more 
flexibility  in  our  ability  to  assign  work  to  the  QC  staff  so  that  all  QC 
Reviews  could  be  completed  in  a  timely  manner.   We  provided  this  training 
by  utilizing  QC  staff  as  instructors,  using  the  Agency's  Staff  Development 
Section  and  use  of  Federal  QC  Staff  from  the  regional  office. 

There  was  and  is  a  great  deal  of  similarity  between  the  AFDC  and  FS 
programs,  both  in  the  QC  policies  and  methods  and  the  eligibility  and 
payment  requirements.   This  made  the  training  much  easier.   We  did  find, 
however,  that  specialized  FS  workers  made  the  adjustment  to  AFDC  easier 
than  specialized  AFDC  workers  adjusted  to  the  FS  program.   This  is  at 
least  partially  explained  by  the  fact  that  FS  workers  had  for  the  most 
part  done  AFDC  reviews  at  some  time  while  AFDC  Reviewers  had  not  done 
FS  reviews. 

QC  Staff  did  not  have  real  serious  problems  adjusting  to  this  change. 
They  have  become  used  to  adapting  to  changes  in  Agency  policy.   QC 
Staff  is  also  used  to  changes  in  the  QC  policies  and  methods  when  new 
review  periods  start  even  in  the  middle  of  review  periods.   Where 
difference  exist  between  the  programs,  initial  confusion  and  some  con- 
tinuing confusion  exist.   These  types  of  situations  include: 
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Income  Exemptions 

AFDC     30  and  1/3 

FS      10%  up  to  30.00 

Child  Care 

AFDC     Maximum  standards 
FS      No  maximum 

Assets 

AFDC     $150  for  the  Grantee  and  $50  for  each  additional  person 
FS       $1500  per  HH 

Shelter  Cost 

AFDC     Consolidated  standard 

FS      As  paid 

(now  as  billed) 

Review  periods  and  verification  periods  involved  in  the  QC  Review: 

AFDC     Focus  on  as  known  as  of  the  review  date  and  in  the  adminis- 
trative period. 
FS      Requires  verification  in  the  review  month  and  averaging  over 
the  certification  period. 

Most  of  the  difficulty  has  been  related  to  failure  to  get  sufficient 
information  in  the  first  contact  to  cover  the  differences  in  the  programs. 
Experience  has  reduced  these  problems  as  workers  have  become  familiar  with 
the  dual  approach. 

Of  all  the  various  situations  and  conditions  that  exist  in  the  integrated 
AFDC-FS  sample,  the  single  most  difficult  and  time  consuming  thing  is  the 
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lack  of  a  uniform  and  combined  review  schedule  and  work  paper  for  use  by 
QC  Staff.   Great  saving  of  time  may  occur  by  using  one  case  record  audit, 
one  home  visit  and  one  verification  effort  for  both  the  AFDC  and  FS  review. 
All  the  information  must  then  be  recorded  twice.   Since  recording  is  com- 
pleted in  either  the  AFDC  case  first  and  the  FS  case  second  or  vice  versa, 
there  is  a  tendency  to  attempt  to  copy  the  one.   As  a  result  some  of  the 
difference  between  programs  are  overlooked  and  must  be  recovered  later. 

While  we  have  not  attempted  to  develop  a  single  recording  format  in  Illinois, 
it  would  seem  to  be  practical.   Such  a  document  instead  of  causing  confusion 
would  focus  on  the  individual  program  differences.   The  elimination  of 
duplicate  recordings  would  further  reduce  the  time  requirements  for  doing 
an  integrated  review. 

We  have  done  some  studies  of  the  time  required  to  do  QC  Reviews  in  the 
various  review  programs.   Several  years  ago  and  continuing  at  intervals, 
a  Performance  Analysis  Section  within  our  Agency  has  been  working  on  a 
performance  standard.   In  this  standard,  the  number  of  hours  of  work 
required  to  complete  a  QC  review  has  been  studied. 

There  is  some  hesitation  in  using  the  figures  developed  so  far  as  there 
is  still  a  considerable  number  of  questions  as  to  the  adequacy  of  the 
standard  developed. 

Our  Performance  Analysis  Section  originally  indicated  an  AFDC  Review  took 
5.8  hours  and  a  FS  Review  took  4.3  hours.   A  internal  study  by  QC  indi- 
cated an  ADC  Review  took  14   hours  and  a  FS  Review  required  12  hours.   A 
subsequent  Performance  Analysis  study  set  a  standard  of  hours  as  follows: 
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COOK 


D.S. 


AFDC 

6.6 

9.2 

FS 

4.9 

6.1 

Combined 

AFDC  and  FS 

7.6 

10.2 

The  difference  between  the  Cook  (metro  Chicago)  and  Downstate  figures  are 
related  to  greater  travel  requirements  is  the  Downstate  area  where  there 
is  a  much  greater  rural  impact. 

Even  allowing  that  there  is  room  for  refinement  in  these  standards,  they 
do  point  out  a  savings  in  time  for  a  combined  review.   This  would  appear 
to  be  33.9%  in  Cook  and  33.3%  for  the  Downstate  area.   The  sample  caseload 
is  not  evenly  distributed  however:   75%  is  in  Cook 

25%  is  in  Downstate 

By  using  the  standard  hours  and  applying  this  to  the  expected  sample 
distribution  total  savings  is  estimated  at  24%  of  the  time  required  for 
'independent  reviews. 


COOK   Standard  Hours 
Cases  X   Hours  =  Required 
=  59-40 
=  4410 
=10350 

=  1630.2 


AFDC 

900  X 

b. 

b 

FS 

900  X 

4. 

9 

Total 

1800  X 

Pure  AFDC 

247  X 

6, 

.6 

Combined 

AFDC/FS 

652  X 

7 

.6 

Pure  FS 

247  X 

4 

.9 

Total  cases 

1146 

D.S.     Standard   Hours 
Cases  X   Hours  =  Required 
300    X    9.2    =  2760 
300    X    6.1    =  1830 
600    X    =  4590 


83 


9.2 


4955.2  218    x   10.2 

1210.3  83    X    6.1 
7795.7      384 


=   763.6 

=  2223.6 

=   506.3 

3493.5 


T . S .   HR . 
1200   8700 
1200   6240 
2400  14940 

330   2392.0 

870   7178.3 

330   1716.6 

1530  11289.2 
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Total 

Reviews     1798 

Savings  Hours 


Staff 


602 


2400 


2554.3 
24 .  7% 
3.2 


096 . 5% 

3650.8 

23 . 9% 

24 . 4°/ 

1.4 

4.6 

Additional  savings  are  involved  in  travel  cost  saved.   Unfortunately,  we 
have  not  as  yet  been  able  to  develop  this  item. 

CONCLUSION: 

There  are  problems  in  developing  a  valid  sample.   These  are  more  the 
problem  of  statistician  than  for  those  of  us  in  the  review  activity  arena. 
Problems  in  training  staff  and  in  helping  staff  adjust  to  the  dual  approach 
do  exist.   The  development  of  accurate  studies  on  the  savings  of  time  and 
money  still  need  to  be  worked  out. 

Our  experience  indicated  the  integrated  sample  is  possible,  if  practical 
and  will  be  most  effective.   We  believe  that  not  only  should  the  AFDC  and 
FS  sample  be  integrated  but  Medicaid  Reviews  can  also  be  integrated.   To  do 
so  and  to  provide  valid  and  effective  results,  QC  staff  must  continue  to  be 
obtained  from  experienced  caseworkers  and  must  have  a  workload  which  allows 
sufficient  time  for  information  findings,  verification,  evaluation  and  deci- 
sion making. 

At  the  Federal  level  all  Agencies  involved  must  provide  assistance  to  the 
states  by: 

1.   Developing  more  uniform  policies  and  procedures  for  determining 
eligibility  and  payment  status. 
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2.  Developing  uniform  QC  procedures  and  methods  and  most  importantly  a 
single  reporting  and  recording  document  to  reduce  duplication  of 
effort. 

3,  Developing  national  standards  of  performance  for  QC  activities  which 
will  enable  State  QC  Units  to  have  a  basis  for  requesting  staff  and 
QC  resources. 
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COLORADO  -  INTEGRATED  SAMPLE  DESIGN 


Presented  by:   George  Kurian,  Statistical 
Analyst 
Colorado  Department  of  Social 
Services 
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AN  OUTLINE  OF  COLORADO 
INTEGRATED  QC  SAMPLING  SYSTEM  * 


On  November  12,  1975,  the  state  received  a  memorandum  from  FNS,  West  Central 
Regional  Office  which  mandated  the  inclusion  of  Public  Assistance  Food  Stamp 
cases  in  the  QC  universe  effective  July,  1976.   Until  that  time  only  NPA  cases 
were  subject  to  the  QC  review  process.   The  new  mandate  increased  the  sample  size 
from  800  to  1200.   This  new  regulation  was  implemented  at  a  time  when  the  State 
could  not  even  complete  the  800  reviews  for  FNS,   In  fact  the  State  never  did 
complete  800  reviews  because  of  the  insufficient  Food  Stamp  QC  review  staff.   Our 
request  for  additional  reviewers  to  implement  the  new  regulation  was  denied  by 
the  State  Legislature,   USDA  threatened  us  with  fiscal  penalty.   This  situation 
lead  us  to  look  at  the  whole  QC  operations  in  a  different  dimension  in  order  to 
bring  maximum  efficiency.   The  new  mapping  of  these  problems  shed  light  on  a 
twofold  answer:   1)  integration  of  the  universes  and  2)  reduction  of  the  number 
of  separate  reviews  required  for  each  one  of  the  QC  programs. 

Sample  surveys  are  often  integrated  when  the  sampling  units  are  areas.   The 
integration  technique  is  widely  used  in  agricultural  surveys.   The  Square  Grid 
Method  (Haynes,  J,D,  (1948)  M.  S,  Thesis):   Central  Square  Grid  (Milne,  A.  (1959), 
Biometrics,  15,  p,  270-297):   Unaligned  Sample  (Quenouille,  M.H  (1949): 
"Problems  in  Plane  Sampling",   Ann,  Math,  Stat,  20,  p.  355-375:   Das.  A.  C, 
'^Two  Dimensional  Systematic  Sampling  and  the  Associated  Stratified  and  Random 
Sampling",  (Sankhya,  10,  p,  95-108):   Extension  of  Latin  Square  (Homeyer,  P,G,, 
and  Black,  C,A  (1946):   "Sampling  Replicated  Field  Experiments  on  Oats  for  Yield 
Determinations",  Proc,  Soil  Sci^  Soc.  America,  11,  p,  341-344):   Lattice  Sampling 
(Yates,  F.  (1960):   and  "Sampling  Methods  for  Census  and  Surveys",  Charles  Griffin 
and  Company,  London,  are  some  of  the  existing  pioneering  works  in  the  field  of 


sampling  integration. 


*  Prepared  by  K.  George  Kurian,  Statistical  Analyst,  Colorado  State  Department  of 
Social  Services,  Denver,  Colorado  80203 

This  paper  was  presented  at  the  National  Conference  on  the  Integrated  Quality 
Control  Systems  in  Washington,  D.C,  February,  1978, 
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The  AFDC,  Medicaid  and  Food  Stamps  case  evaluation  processes  which  comprise 
the  three  QC  review  operations,  can  be  integrated  since  the  eligibility  for  each 
one  of  these  programs  depends  mainly  on  income  and  resources.   Age,  disability, 
marital  status,  presence  of  children,  school  attendance  etc.  are,  I  would  call, 
technical  eligibility  conditions.   However,  if  these  technical  conditions  are 
met,  the  income  and  resources  determine  an  applicant's  eligibility  for  one  or 
all  three  program  areas.   Thus,  basically  the  same  people  receive  the  benefits 
under  these  three  programs.   If  this  is  the  case,  can't  we  design  a  sampling 
system  which  would  maximize  the  selection  of  samples  from  the  multiple  program 
beneficiary  segments  of  the  three  universes?  The  burden  of  designing  such  a 
system  rests  with  the  states,  since  one  federal  agency  seldom  knows  what  the 
other  agency  does,  and  the  state  is  the  point  of  convergence  of  all  the  trans- 
mittals from  the  federal  agencies. 

Our  analysis  of  the  three  populations  as  intersecting  sets  helps  us  to  classify 
the  cases  which  are  included  in  the  three  QC  processes  by  their  levels  of  program 
participation.   Seven  possible  strata  will  emerge  through  this  classification 
procedure.   These  are  depicted  below, 

SIX  MONTH  UNIVERSES 
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A  =  AFDC 

B  =  Medicaid 

C  =  Food  Stamps 

Each  of  the  three  QC  universes  is  given  by  the  union  of  four  disjoint  sets. 
The  AFDC  universe  is  given  by  1,  A- [(A  OB)  U  (AA  C)-  (A  0  B  DC)] ,  2.  [(AHB)- 
(AnBHC)],  3.  [(AnC)—  (AHBnC)],  and  4   (AABHC).   The  Medicaid  Universes  is 
given  by  5.  B-[(AnB)  U  (BO  C)  -  (AA  B  QC)]  ,  6.  [  (AO  B)-  (An  B  AC)] ,  7.  [  (Bn  C)- 
(AABHC)],  and  8.  (Ar»Br\C).   The  Food  Stamp  Universe  is  given  by  9,  C-[(BnC) 
U  (AnC)-  (AOBnC)],  10,  [(BnC)-  (AABHC)],  11.  [(AOC)-  fAf^BfVC)],  and  12. 
(AHBOC). 

In  December  1975,  I  discussed  these  ideas  in  general  with  the  DHEW  Regional 
Office,  USDA  Regional  Office  in  Dallas  and  FNS  Office  in  Washington,  D.C.   The 
DHEW  Regional  Office  offered  encouragement  whereas  the  USDA  Office  began  to  pour 
questions  and  more  questions  on  us.   At  this  stage,  I  discussed  this  integration 
idea  with  Professor  Herbert  Solemon  of  Stanford  University,  the  USDA'S  Consultant, 
who  talked  to  the  USDA  Washington  Office  and  convinced  them  of  the  advantages  of 
the  integrated  design.   In  the  early  part  of  1976,  we  discussed  these  ideas  with 
Dr,  Victor  Kugjasky  and  Mr,  William  Hickey  of  DHEW,  Washington,  D.C.   They 
advised  us  to  apply  for  a  Research  and  Demonstration  Project  since  DHEW  is  looking 
for  ways  and  means  to  reduce  the  impact  of  all  regulations  on  the  states  and  to 
minimize  the  costs.   We  applied  for  an  1115  project  in  February,  1976,   This 
application  was  approved  in  November  of  1977,  and  the  project  started  on  December 
1,  1977,   The  lack  of  the  specifics  of  the  anticipated  Food  Stamp  regulations  and 
MEQC  changes  have  slowed  the  progress  of  the  project  work  at  this  time.   We  hope 
to  complete  this  project  by  July,  1979, 
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We  decided  to  integrate  the  AFDC  and  Food  Stamp  QC  Programs  effective 
July  1,  1976  because  of  the  anticipated  delay  in  getting  the  1115  project 
proposal  approved.   Only  such  an  approach  could  give  us  a  viable  means  of 
completing  the  mandated  1200  FS-QC  reviews.   The  design  approved  by  USDA  and 
DREW  is  described  below. 

1.  Select  the  AFDC  Samples  according  to  the  instructions  given 
in  the  AFDC  Manual. 

2.  Estimate  the  Non-AFDC  and  AFDC  Case  loads  for  six  months. 
Determine  the  number  of  samples  to  be  selected  from  the  Non- 
AFDC  stratum  and  the  interval. 

3.  Select  1  samples  from  the  Non-AFDC  stratum  using  the  interval 

i 
computed  above. 

4.  Determine  the  sample  size  for  the  AFDC  group  by  computing 

m  =  M  1   ,  where 

i  J, i 

L 
i 

m  =  the  AFDC-FS  Sample  size  for  the  i-th  month, 

i 
M  =  the  actual  AFDC-FS  population  size  for  the  i-th  month, 

i 
L  =  the  actual  Non-AFDC-FS  population  size  for  the  i-th  month,  and 

i 
1  =  the  number  of  Non-AFDC-FS  Samples  selected  for  the  i-th  month. 

i 

5.  Obtain  the  FS  participation  information  on  each  one  of  the  cases  on 
the  AFDC  sample  listing. 

6.  Subsaraple  from  the  AFDC  cases  participating  in  FS,  the  list  obtained 

at  step  5,  the  required  m  samples. 

1 

After  implementing  this  system  in  July,  1976,  ve  did  a  time  study.   The 
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results  from  this  study  are  given  below. 


AVERAGE  TIME  (Hours)  PER  COMPLETED 

CASES 

AFDC 

AFDC  with  FS 

NON-AFDC-FS 

MED 

Case  Record  Analysis 

1,87 

- 

1.05 

1.38 

Home  Visit 

1.24 

- 

1.38 

1.02 

Travel 

1.44 

- 

4.04 

1.44 

Case  Completion 

4.10 

2.95 

4.59 

2.07 

TOTAL 

8.65 

2.95 

11.06 

5,91 

Reviews  Required 

800 

430 

770 

275 

No  explanation  is  necessary  to  notice  the  cost  savlngs-approximately  5  reviewers. 

As  we  mentioned  above,  the  FS  participation  of  the  AFDC  Cases  were  determined 
manually  from  the  FS-11  (Counter  Issuance  Record)  and  FS-18  (Mall  Issuance  Log). 
We  also  Issued  a  County  Letter  so  that  the  Counties  could  provide  us  the  In- 
formation whether  a  participating  case  Is  or  Is  not  an  AFDC  headed  household. 
Since  we  were  anticipating  the  approval  of  the  1115  project,  no  extenpive  train- 
ing to  the  County  Personnel  was  provided,.  Mipclapsification  of  cases  by  the 
Counties  reduced  the  number  of  matched  FS  and  AFDC  casep  and  affected  the  cost 
benefit. 

When  we  implemented  this  design,  the  reviewers  were  not  cross  trained  In 
all  three  QC  programp.   The  FS  reviewer,  and  the  AFDC  and  Medicaid  reviewer 
team  still  exist.  Because  of  all  these  problems,  the  cost  savings  we  have  shown 
in  the  above  table  are  only  minimal. 

Even  though  it  is  not  possible  to  isolate  the  effect  of  this  mini-integration 
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from  the  other  administrative  actions, which  might  have  impacted  the  error  rates 
during  the  QC  periods,  specified  below,  we  are  showing  below  the  QC  error  rates 
in  the  AFDC  and  FS  programs. 


QC  CASE 

ERROR  RATES  -  POSITIVE 

PERIODS 

AFDC 

7o 

AFDC 
FS 

7o 

NON-AFDC 
FS 

7„ 

TOTAL 
FS 

7„ 

July  -  Dec,  1975 

18.5 

47.5 

Jan.  -  June  1976 

18.7 

41.4 

July  -  Dec.  1977 

13.3 

25.6 

31.4 

32.0 

Jan.  -  June  1977 

10.7 

24.8 

30.8 

22.7 

The  1115  project  we  are  working  on  now  will  eliminate  the  FS  reporting 
problems  and  integrate  the  AFDC,  FS  and  Medicaid  Quality  Control  Systems.   In 
order  to  eliminate  the  FS  reporting  problems,  we  have  tentatively  designed  a 
new  FS-9  form,  an  authorization  and  participation  report.   This  form  would 
replace  the  existing  FS-9,  FS-18  forms.   The  work  load  at  the  County  level  will 
be  reduced.   This  new  form  will  collect  the  head  of  the  household's  case 
number,  the  public  assistance  category  code  and  the  Social  Security  Number. 
This  information  will  help  us  to  detect  duplicate  participation  either  in  the 
same  county  or  in  some  other  county,  and  this  form  will  provide  the  data  for  the 
FNS-256  report  which  in  turn  will  result  in  the  better  management  information 
for  the  FS  Division  and  the  Department  of  Social  Services.   A  copy  of  this  form 
is  attached  here  to.   We  cannot,  however,  implement  this  form  or  begin  to  train 
the  County  personnel  because  of  the  lack  of  finalization  of  the  FS  regulations  at 
present  and  also  because  of  the  pending  determination  of  the  implementation  date. 
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As  we  all  know,  the  Medicaid  QC  is  going  to  change.   Who  will  be  included 
and  excluded  from  the  universe  is  unknown  at  this  time.   In  Colorado,  an  AFDC 
case  may  have  more  than  one  individual  eligible  for  tfedicaid.   In  the  universe, 
should  we  count  them  individually  or  count  the  case  as  one  case?  The  eligi- 
bility for  assistance  is  mainly  based  on  income  and  resources.   Some  of  the 
members  in  a  family  may  or  may  not  meet  some  of  the  technical  conditions  and, 
consequently,  they  do  not  qualify  for  Medicaid,   If  only  one  individual  within 
a  case  is  included  in  the  frame  and  if  this  one  is  selected  for  review,  the 
reviewer  has  to  make  an  initial  determination  of  whether  or  not  the  whole  family 
is  eligible  based  on  income  and  resources.   This  assumes,  of  course,  that  this 
individual  meets  the  technical  conditions.   What  will  be  the  new  review  forms 
and  the  contents  of  the  6-month  reports?  We  all  hope  these  concerns  will  be 
answered  soon. 

Once  we  get  the  FS  regulations  finalized  and  the  Medicaid  Universe  clearly 
defined,  the  project  can  start  work  on  the  computer  programs  needed  to  automate 
the  FS-9  and  create  a  composite  record  for  each  case  to  be  included  in  the 
universe.   This  composite  record  will  contain  the  information  on  the  multiple 
program  participation  and  the  benefits  received  under  each  one  of  the  programs. 
The  universe  will  then  contain  only  these  records  from  which  samples  will  be 
selected  for  reviews  in  the  three  QC  areas.   The  sample  listings  would  also  carry 
the  multiple  program  participation  information. 

In  the  AFDC-QC,  we  can  restrict  the  universe  to  the  main  payrolls.   All  the 
main  payrolls  will  be  in  the  state  office  and  ready  for  sampling,  using  computer, 
by  the  end  of  the  2nd  week  within  a  month.   We  can  restrict  the  universe  to  the 
main  payrolls  only  if  certain  conditions  are  satisfied.   If  HCFA  and  FNS  have  the 
same  type  of  rules,  we  will  be  able  to  get  all  the  samples  selected  in  the  second 
weekend  and  ready  for  assignments  to  the  reviewers  in  the  3rd  week.   Thus, 
the  reviewers  will  have  two  to  three  additional  weeks  to  work  on  these  cases. 


We  are  planning  to  Integrate  the  review  forms,  i.e.,  the  work  sheets  and 
data  sheets,  to  the  maximum  extent  that  we  can.   The  changes  in  the  MEQC  and 
FS-QC  may  be  a  blessing  in  disguise.  We  also  plan  to  have  only  one  review  data 
file  which  can  provide  the  information  required  to  generate  all  6  month  federal 
reports  and  the  monthly  status  reports. 

Since  a  case  will  be  reviewed  for  multiple  QC  programs,  we  may  be  able  to 
get  satisfactory  estimates  by  different  strata  of  the  amount  of  money  that 
federal,  state  or  local  government  paid  in  error.   The  strata  are  as  follows: 
(1)  AFDC  only,  (2)  AFDC  and  FS,  (3)  AFDC  and  MED,  (4)  AFDC,  FS  and  MED,  (5)  FS 
and  MED,  (6)  FS  only,  and  (7)  MED  only.   Corrective  actions  can  be  concentrated 
on  those  areas  where  significant  amounts  of  money  are  wasted.  Also,  any  cor- 
rective action  taken  in  a  stratum  consisting  of  multiple  programs  can  kill  two 
or  three  birds  at  a  time.  This  can  save  the  state  agency  the  time  they  spent  in 
taking  different  corrective  actions  aimed  at  the  same  group  of  clients  or  in  going 
against  the  groups  where  the  benefit  from  the  corrective  action  will  be  insignif- 
icant. 

When  a  reviewer  finds  there  exists  an  error,  he/she  sends  a  QC-10  report  to 
the  county  requesting  that  the  situation  be  corrected.  The  reviewer  needs  to 
send  only  one  QC-10  report  to  the  county  for  all  error  cases  from  the  multiple 
participation  stratum.  The  county  can  take  action  to  correct  the  situation  in 
multiple  program  areas  thus  resulting  In  some  savings  in  time  at  the  county  level. 

As  I  mentioned  earlier,  there  exists  a  team  of  reviewers  for  Public  Assistance 
QC  and  another  team  for  FS-QC.   Because  of  this  situation,  the  supervisors  in  the 
QC  section  do  not  have  equal  work  loads.   Under  this  full  integration  of  the  three 
QC  programs,  where  the  reviewers  will  be  assigned  cases  from  all  the  three  QC 
areas,  the  units  within  the  QC  section  can  be  rearranged  so  that  the  supervisors 
will  have  equal  work  loads.  Also,  the  reviewers  will  not  have  any  good  reason  to 
complain  that  a  FS  case  is  tougher  than  an  AFDC  case  or  vice-versa. 
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Since  all  reviewers  will  be  doing  all  types  of  case  reviews,  we  may  be  able 
to  outstation  reviewers  to  three  or  four  towns  in  the  State  so  that  the  review- 
ers from  the  central  office  do  not  have  to  travel  all  the  way  to  the  distant 
towns  to  review  the  cases  from  those  areas.   This  will  cut  down  the  travel  time 
possible  overhead  expenditures. 

We  all  know  that  people  seldom  welcome  and  appreciate  changes.   That  is 
simply  human  nature.   The  reviewers  do  criticize  the  integration  approach  because 
they  think  that  they  will  have  to  learn  the  rules  and  regulations  for  each 
program  and  keep  abreast  of  all  the  rule  changes  for  each  program.   This  is  true 
but  the  reviewers  who  are  now  reviewing  AFDC  and  AFDC-FS,  though  they  also 
complained  prior  to  the  present  mini-integration,  have  since  changed  their  at- 
titudes.  The  junior  most  reviewers  think  that  they  may  be  laid  off  because  of 
fewer  staff  requirements.   However,  this  is  highly  improbable  since  more  QC  require- 
ments may  be  coming  down  and  because  program  divisions  seem  to  always  be  needing 
more  people  to  administer  their  programs  in  providing  ever  expanding  services  to 
needy  people. 

Another  major  problem,  especially  in  the  states  where  the  state  supervises 
and  the  county  administers,  is  to  get  the  data  correct.   If  wrong  FS  case  numbers 
are  provided  by  the  counties,  the  matching  of  the  cases  becomes  impossible  and, 
consequently,  will  reduce  the  cost  efficiency.  We  hope  to  minimise  these  data 
errors  by  providing  extensive  training  to  the  county  staff. 
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SAMPLE  DESIGN 

None  of  the  three  agencies  is  imienable  to  a  drastic  change  in  their  sampling 
plans.   Consequently,  we  had  to  design  a  sampling  scheme  which  would  satisfy 
everybody.   This  plan  may  not  be  the  best  cost  benefical  or  an  optimal  plan. 

We  plan  to  select  the  AFDC  samples  according  to  the  AFDC  Manual.   The 

selection  of  a  systematic  sample  from  the  AFDC  frame  would  provide  us  a  stratified 

proportional  sample.   Since  the  sampling  fraction  is  uniform,  the  design  would 

provide  us  a  self  weighting  sample.   Let  n  ,  n   ,  n   and  n   be  the  number  of 

11   12   13      14 
samples  selected  from  the  strata  of  sizes  N  ,  N  ,  N   and  N  .   The  first 

11   12   13      14 
subscript  denotes  the  QC  program  and  the  second  subscript  denotes  the  strata. 

The  strata  definitions  are  given  on  page  3 . 

FNS,  as  per  their  memorandum  of  Sept  7,  1976,  wants  only  a  proportional 

representation  of  AFDC  and  Medicaid  cases  in  their  QC  sample.   Let  N  ,  N  , 

31   32 
N   and  N   be  the  strata  sizes  in  the  Food  Stamp  Universe  and  the  number  of 

33      34 
samples  which  would  be  selected,  based  on  their  manual  instructions,  from  these 

strata  be  n   ,  n   ,  n   and  n   .   These  sample  sizes  are  determined  in  such  a 

31   32   33      34 
way  that  m   =  m   x  N   for  each  i,  where  m   =m   +m   +m   +m.N   is 
3i    3.    3i  3.    31    32    33    34    3. 

N 
3. 

defined  in  the  same  manner. 

N  ,N  ,N  ,N  ,m  ,m  ,m  ,  and  m   are  defined  as  above  for  the 
21   22   23   24   21   22   23      24 
Medicaid  Universe  since  HCFA  has  changed  its'  universe  from  the  paid  claims  to 

eligibles  and  eliminated  the  condition  to  select  one-half  of  the  sample  from 

the  institutions  stratum.   This  may  change  their  sampling  criteria  and  we  would 

be  able  to  fix  m   =  m   x  N   . 
2i    2.    2i 

N 
2, 

Qn  January  31,  1978,  Mr,  Manny  Schwartz  informed  me  that  FNS  and  HCFA  have 
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agreed  with  him  to  review  all  available  cases  in  a  stratum  instead  of  subsampling 

or  taking  additional  samples  to  meet  the  condition  of  proportional  representation. 

In  light  of  our  discussion  on  that  matter,  I  will  henceforth  ignor  the  last  two 
paragraphs  on  page  4. 


N   =N,N   =N,N   =N,N   =N   =N. 
12    22   13    33   32    23   14    24    34 

Now  let  us  state  the  constraints  imposed  on  the  sample  sizes. 

n   +n   +n   +n   =  800 

11  12  13  14 
m   +m   +m   +m   =  300 

21  22  23  24 
m   +m   +m   +m   ^  ^^^^ 

31    32    33    34 


Theorem  1 

The  system  of  equations  A  x  =  g  is  consistent  if  and  only  if  AA"g  =  g 
Theorem  2  ' 

The  BAS  (Best  Approximation  Solution)  to  the  inconsistent  system  of 
equations  A  x  =  g  is  Xq  where  Xq  =  A"g, 

Rewriting  the  constraint  equations  as 

a  +a  +a  +a  »  800 

12  3         4 

b+b+ka+ka=   1200 

12  13         2     4 

c+ka+kb       +ka-300 

1         3     2         4        2         5     4 

help  us  to  write  these  in  the  following  matrix  notation. 


11110  0  0 

0  0  k  k  1  1  0 

1  2 

0  k  0  k  0  k  1 
3   5   4 


800 

1200 

300 


l^J 


112  - 


Let  A  denote  the  coefficient  matrix.  A' A  is  a  S3mimetric  matrix  and  A''= 

AJ .  Thus,  there  does  exist  a  BAS  solution  which  will  be  a  function  of 

lA'AI 

k  ,  k  ,  k  ,  k  and  k  .   Different  types  of  plans  could  be  obtained  choosing 
12   3   4      5 

the  values  of  these  variables.   However,  we  would  be  using  an  inspection  method 

to  get  the  solutions. 

Since  we  know  that  the  above  set  of  equations  has  finitely  many  solutions 

and  Mr.  Schwartz  has  the  agreement  from  HCFA  and  USDA  that  they  do  not  need 

any  more  of  the  proportional  representation  from  each  one  of  the  respective 

strata  in  their  samples,  we  will  review  n   and  n   samples  for  AFDC  and  FS 

13      14 

and  n  ,  m   and  n   samples  for  AFDC/FS  and  Ifed. 
12   32      14 

What  we  discussed  above  should  be  the  end  product  at  the  end  of  the  6 
month  QC  period.  The  universe  siaes  are  estimates.   Samples  are  selected 
every  month.  Every  month  these  ciraumstances  m«ke  the  sample  selection 
process  a  little  more  complicated.  We  would  describe  the  monthly  selection 
processes  for  the  three  QC  programs  below. 

Let  A  be  the  estimated  universe  for  the  AFDC-QC,   The  interval  for  6 

months  is  I  =  A .   A  stands  for  the  actual  population  size  and  a  stands  for 

800 
the  number  of  samples  selected  in  the  following  table.  Also  a   =  A  ,  and 

i.   i_ 

I 
a   =  A   . 
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TABLE  1 


DISTRIBUTION  OF  THE  SELECTED 
AFDC  SAMPLES 


MONTHS 

AFDC 

AFDC,  MED 

AFDC,  FS 

AFDC,  MED 
FS 

TOTAL 

I 

^1 

^2 

^3 

S4 

s. 

^1 

^12 

^13 

^14 

^. 

2 

^1 

^2 

^3 

^4 

s. 

^21 

^22 

^23 

^24 

"2. 

3 

^1 

^2 

^3 

^4 

s. 

^31 

^32 

^33 

^34 

^3. 

4 

^1 

^2 

\3 

\4 

\. 

^41 

^2 

^43 

^4 

^. 

5 

Si 

^2 

^3 

^4 

s. 

^51 

^52 

^53 

^54 

^5. 

6 

\l 

^2 

^3 

64 

S. 

^61 

^62 

^63 

^64 

^6. 

TOTAL 

^•1 

A.  2 

A.  3 

^•4 

A 

^•1 

a. 2 

a.3 

^•4 

a  =  800 
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TABLE  2 


DISTRIBUTION  OF  EXPECTED  AND 
AVAILABLE  FOOD  STAMP  SAMPLES 


MONTH 

FS 

FS,  MED 

FS,  AFDC 

FS,  MED 
AFDC 

TOTAL 

1 

^U 

^12 

^13  =  ^3 

^14  =  ^4 

^1. 

^11 

^12 

^13,  ^13 

^14,  ^14 

h. 

2 

^21 

F 

^23  =  ^23 

^24  =  ^24 

^2. 

^21 

^22 

^23,  ^23 

^24,  ^24 

h. 

3 

^31 

^32 

^33  =  ^33 

^34  =  ^4 

h. 

^31 

^32 

^33,  ^33 

^4,  ^34 

h. 

4 

^1 

^42 

^3  =  \3 

^44  =  \4 

h. 

^41 

^42 

^43,  ^3 

^44,  ^44 

h. 

5 

^51 

^52 

^53  =  ^3 

^54  =  ^4 

^5. 

^51 

^52 

^53,  ^53 

^54,  ^4 

h. 

6 

^61 

^62 

^3  =  ^3 

^64  =  ^4 

h. 

^61 

hi 

^63,  ^63 

^64,  ^64 

h. 

TOTAL 

^•1 

^•2 

F.3=  A.3 

F   =  A 
4     4 

F 

f-.l 

f-2 

^•3,  ^'3 

f.,   a 
4,   4 

f  =  1200 

115  - 


At  the  time  of  sampling  we  will  know  the  values  of  F   ,  F   ,  F   ,  F   , 

1.    2,    3.   4, 

F  ,  and  F  ,  whereas  F  has  to  be  estimated  in  advance  to  determine  the 
5.       6. 

interval  1  =  F ,   In  the  first  month  we  would  select  f   =  F     samples. 

1200  1.     1. 


From  each  stratum  we  would  select  f   =f   xF   ,j=l,  ,.,,4.   The 

Ij    1.   Jj. 

F 

1. 

number  of  samples  which  would  be  selected  in  the  second  month  is  f   = 

2. 


Tf   +  F       ^   "l 


The  Strata  Samples  will  be  given  by  f   =  f   x  F      xhus 

2j    2.    6j 
F 
6. 
at  the  end  of  a  6-month  period  we  would  get  f  =  XI-     ^        samples. 

i    i. 


Review  all  the  a   and  a   samples  for  AFDC  and  FS.   Then  f   -  (a   +  a   ) 
i3      14  1.     i3    i4 

will  be  selected  from  strata  1  and  2  systematically  and  reviewed  for  FS. 
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TABLE  3 


DISTRIBUTION  OF  EXPECTED  AND 
AVAILABLE  MEDICAID  SAMPLES 


MONTHS 

MED 

MED,    AFDC 

MED,    FS 

MED,   AFDC 
FS 

TOTAL 

1 

^11 

m 

11 

^12  =  \2 
"^12,    ^2 

M        =   F 
^3        ^12 

^3,    ^12 

M       =   F^,    =  A,, 
14          14          14 

m           *•          f 
14,      14,      14 

^1. 

2 

^21 

tn 
21 

^22  =  ^2 
^2,    ^22 

M        =   F 
23          22 

"^23,    ^22 

^24  =  ^4  =  ^24 
"^24,    "24,    ^24 

^2. 

3 

M 
31 

m 
31 

^32  =  ^2 
"^32,    ^2 

M        =   F 
33        ^32 

"^33,    ^32 

M         =    F         =    A 
34       ^  34          34 

"^34,    '34.    ^4 

M3 

4 

^1 

m 

41 

^42  =  \2 
\2,    ^2 

\3  =  ^2 
^3,    ^43 

^4  =  ^4  =  ^4 
^4,    '44,    ^4 

5 

^51 
"51 

^52  =  ^2 
^2,    ^52 

M        =   F 
"^53        ^52 

^3,    hi 

^4  =  ^4  =  ^4 
"^54,    '54.    S4 

^5. 

"5. 

6 

^61 

m 

61 

^62  =  ^2 
^62,    ^2 

^63  =  ^62 

m          f 
63,      62 

64          64          64 

m          a          f 
64,      64,      64 

^6. 
"6. 

TOTAL 

^•1 
m.  ^ 

M.2  =  A^ 

^•6  =  ^-6 
"^•6,    ^-2 

^•4  =  ^-4  =  ^-4 
"•4,    ^-4,    ^-4 

M 

m  =   300 
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We 


have  to  select  300  samples  for  the  Medicaid  Qc.  A  systematic  sample  of 


this  sise  can  be  selected  by  choosing  an  interval  p  °  M  ,  where  M  is  the 

300 

estimated  population  siee  for  6  months.   M   and  m   can  be  defined  in  the  same 

i.      ij 

way  as  we  defined  the  similar  variables  above. 

As  before,  we  would  compute  m   -  m   -  (a   +  f   +  a  )  and  select  them 

il    i.     i2    i2    i4 

systeraetically  from  stratum  1. 

In  both  the  FS  and  Medicaid  tables,  I  have  shown  all  the  f   and  m   sample 

ij      ij 

sizes  to  indicate  that  they  will  be  the  sizes  if  we  use  proportional  allocation 
to  get  uniform  sampling  fractions. 

The  following  3  tables  would  exemplify  the  selection  processes.  The 
population  sizes  used  here  are  estimates. 
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TABLE  4 

DISTRIBUTION  OF  SELECTED 
AFDC  SAMPLES 


M3NTH 

AFDC 

AFDC,  MED 

AFDC,  FS 

AFDC,  FS 
MED 

TOTAL 

1 

1000 
4 

2000 
10 

7000 
35 

20000 
98 

30,000 
147 

2 

1500 

7 

2400 
12 

6100 
30 

19000 
93 

29,000 
142 

3 

1200 
6 

2300 
11 

6500 
32 

21000 
103 

31,000 
152 

4 

1400 

7 

2100 
10 

4000 
19 

20500 
101 

28,000 
137 

5 

1500 

7 

1500 
8 

7000 
34 

22000 
108 

32,000 

157 

6 

500 
2 

1500 

8 

8000 
39 

20,000 
98 

30,000 
147 

TOTAL 

7100 
33 

11800 
59 

38600 
189 

122,500 
601 

180,000 
882 

Let  the  AFDC  Universe  for  6  months  be  =  30,000  x  6 

In  order  to  select  800  x  1.10  =  880  samples,  allowing  for  drops, 

the  interval  =  30,000  x  6  =204 
880 
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TABLE  5 

DISTRIBUTION  OF  EXPECTED  AND 
SELECTED  FS  SAMPLES 


MONTHS 

FS 

FS,  MED 

FS,  AFDC 

FS,  MED 
AFDC 

TOTAL 

20000 

3000 

7000 

20,000 

50000 

1 

35 

98 

96 

14 

34 

96 

240 

19000 

4900 

6100 

19,000 

49000 

2 

30 

93 

91 

24 

29 

91 

235 

18000 

500 

6500 

21,000 

51,000 

3 

32 

103 

87 

26 

32 

101 

246 

21000 

6500 

4000 

20,500 

52,000 

4 

19 

101 

101 

31 

19 

99 

250 

18000 

3000 

7000 

22,000 

50000 

5 

86 

15 

33 

106 

240 

17000 

3000 

8000 

20,000 

48000 

6 

82 

14 

39 

96 

231 

113000 

25,900 

38600 

122,500 

300,000 

TOTAL 

543 

124 

186 

589 

1442 

Let  the  FS  Universe  for  6  months  be  300,000. 

An  interval  of  300.000  =  208  would  be  required  to 
1440 

select  1200  x  1.2  =  1440  samples. 

Please  notice  that  1,457  cases  are  available  for  FS, 
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TABLE  6 

DISTRIBUTION  OF  EXPECTED  AND 
SELECTED  MI'DICAID  SAMPLES 


MONTHS 

MED 

MED,  AFDC 

MED,  FS 

MED,  AFDC 
FS 

TOTAL 

1 

17,000 
19 

2,000 

10 

2 

3,000 

14 

4 

20,000 

98,96 

23 

42,000 
48 

2 

18,000 
20 

2,400 

12 

3 

4,900 
24 

6 

19,000 

93,91 

21 

44,300 
50 

3 

16,000 
19 

2,300 

11 

2 

5,500 

26 

6 

2], 000 

103,101 

25 

44,800 
52 

4 

19,000 
21 

2,100 

10 

3 

6,500 

1 

7 

20,500 
101,99 
14 

48,100 
55 

5 

15,000 
17 

1,500 

8 

1 

3,000 

15 

4 

22,000 

108,106 

25 

,41,500 
47-''' 

6 

17,000 
19 

1,500 

8 

2 

3,000 

14 

4 

20,000 

98,96 

23 

41,500 
48 

TOTAL 

102,000 
115 

11,800 

59 

13 

25,900 

124 

31 

122,500 
601,589 
141 

262,200 
300 

Let  the  Medicaid  Universe  for  6  months  =  262,200,   To  pelect  300 
samples  syFtematically,  we  fix  the  interval  =  262,200  «■  874, 


Please  note  that  the  total  number  of  samples  available  =  899,   Here  sub- 
.  sampling  is  necessary  to  exclude  599  samples;  otherwise  the  cost  woiiM  be 
599  X  3  =  1,797  man  hours  at  a  cost  of  $10,  per  hour  «  $17,970. 

Thus,  the  best' plan  tc  reduce -the  cost  of  reviews  can  he  obtained  through 

the  following  allocation  plan  for  FS  and  MEDICAID; 
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FOOD  STAMPS  -  QC 
For  j  =  3,4 


n   =  a   irrespective  of  the  relation  between  f   and  a 
ij    ij  ij      ij 

and  n   +n   =n   -n   -n 

il    i2    i.    i3    i4 

MEDICAID  -  QC  (Assuming  300  Samples) 

n   =a   -(a   -m)ifm   ^  a 

12  12     12    12      i2    i2 

n   =f   -(f   -m)ifm  ^f 

13  12     12    13      13    13 

n   =a   -(a   -m)ifa   »m 

14  14     14    14  •    i4    14 

and  n   =n   -(n   +n   +n) 
11    i.     12    13    14 

When  the  population  is  created  and  the  strata  sizes  are  known,  it  would  be 
necessary  to  decide  whether  we  should  combine  some  of  the  strata  or  not.   If 
a  stratum  population  size  is  such  that  we  cannot  get  at  least  one  sample,  there 
would  not  be  any  need  to  keep  such  a  stratum  separate. 
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CHANGE  IN  THE  EXISTING  MEDICAID  QUALITY  CONTROL  SYSTEM 

1-^—     I         ■  ■-    ■■■     L.^         ■  — ■■   ■        li  J  ■_!    ■!—    ■■■■»■■  ■^-.  a.         ■  >       I       !■■■    I  ■-  I  I    .  ■  M 

On  February  6,  1978,  the  State  received  HCFA-AT-78-9,  dated  January  31, 
1978.   Owing  to  the  late  receipt  of  the  AT  (received  only  on  February  6,  1978), 
we  could  not  make  necessary  changes  in  the  above  discussions  to  reflect  the 
current  changes.   Discussions  with  Mr.  Allen  Bazar  of  HCFA  on  February  8,  1978, 
lead  me  to  believe  that  this  AT  does  not  have  any  adverse  effect  on  the  In- 
tegrated Sampling  Design. 

Since  Colorado  is  a  209  b  State,  this  AT  gives  us  the  permission  to  review 
all  the  800  AFDC  Samples  for  >fedicaid  also  and  to  substitute  the  SSI  Medicaid 
Samples  which  would  ordinarily  be  selected  from  the  Medicaid  Strata  1  and  3  with 
the  SSA-QA  Samples.   This  substitution  is  limited  to  one-half  of  275  non-AFDC 
Medicaid  Samples. 

Based  on  this  new  AT,  let  us  re-write  the  sample  constraints  Cplease  refer 
to  page  11). 

n   +n   +n   +n   =  800 

11  12    13    14 

m   +  m    =275 
21    23 

m   +m   +m   +m   =  1200 
31    32    33    34 

Rewriting  this  in  order  to  bring  the  strata  sample  relationships,  we  would  get 

a  +a  +  a     +  a      =  800 

12  3    4 

b     +kc     +ka     +ka     =■   1200 
112  2   3  3  4 

c     +  c      =  275 
1         2 

If  we  are  given  the  values  of  k   k  and  k   we  can  obtain  the  HAS,  though  none 

1,   2      3, 

of  the  agencies  will  change  the  sample  size. 

The  sample  selection  procedures,  following  the  approach  mentioned  on  pages 

13-16,  can  be  described  in  short,  in  the  following  order. 
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1.  Select  AFDC  and  Non-AFDC  Medicaid  Samples. 

2,  Select  (1200-Food  Stamps  Samples  available  from  AFDC  and  Medicaid 
Strata)  samples  from  the  Food  Stamp  only  strata  systematically. 

Depending  on  the  number  of  SSI  Samples  which  will  be  selected  from  the 
Non-AFDC  Medicaid  segment  and  the  cost  benefit  which  can  be  derived  from  the 
substitution  provision,  the  management  can  make  the  decision  whether  or  not 
to  utilize  some  of  the  provisions  given  to  the  209  b  State  under  the  AT. 
SUMMARY 

In  summary  the  advantages  which  can  be  gained  through  integration  and 
unification  of  the  three  QC  procedures  outweigh  the  troubles  in  getting  the 
work  organized  for  the  project  and  training  the  reviewers  all  the  three  QC 
programs.   If  HCFA  and  USDA  give  us  the  permission  to  demonstrate  that  the 
sampled  population  will  be  in  no  way  different  from  the  target  population,  we 
would  be  able  to  obtain  greater  speed  to  complete  the  reviews  in  time  with 
reduced  cost  per  review.   The  sampled  population  will  consist  of  the  first  15 
days  of  participation  or  the  Medicaid  eligibles  on  the  roll  in  the  first  15  days 
in  a  month.   For  a  State  like  Colorado  this  would  mean  a  minimum  of  54  man-weeks 
in  a  period.  This  gain  een  offset  the  additional  reviewer  requirements  In  the 
MQC,  MEQC  negative,  and  AFDC  QC  negative. 

It  is  our  pleasure  to  extent  our  thanks  and  gratitude  to  the  officials  in 
SSA,  HCFA  and  USDA;  without  their  encouragement  the  integration  idea  would  have 
been  nipped  in  the  bud.   We  also  extend  our  special  thanks  to  the  Family 
Assistance  Studies  Staff  and  the  Project  Officer  who  always  show  great  interest 
in  seeing  that  all  the  agencies  are  coordinated  and  in  providing  us  all  the  help 
they  can  to  eliminate  all  the  impediments  which  slows  the  progress  of  the  project. 
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NEW  YORK  -  INTEGRATED  SAMPLE  DESIGN 


Presented  by:   Charles  Kavanaugh,  Assistant  Director 
Bureau  of  Audit  Operations 
New  York  Department  of  Social  Services 
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Integration  of  the  AFDC  and  F/S  Quality  Control  Samples 

In  order  to  understand  the  rationale  for  our  integrated  sampling  plan,  I 
will  explain  how  New  York  State  administered  the  AFDC  and  Food  Stamp 
Programs  and  the  problems  we  in  Quality  Control  must  cope  with. 

We  have  a  State-supervised,  county-administered  system.   The  State  is 
comprised  of  57  counties  and  New  York  City.   There  are  approximately 
350,000  AFDC  cases  and  615,000  F/S  cases.   Naturally  our  six-month 
sample  requirements  are  1,200  positive  cases  in  each  program.   New  York 
City  is  responsible  for  approximately  70  percent  of  the  total  caseload 
in  both  programs.   I  might  also  mention  that  at  the  present  time  we  do 
not  have  a  central  eligibility  file  for  any  social  services  program.   We 
expect  to  have  this  capability  in  1980  when  our  Welfare  Management  System 
is  fully  developed.   Presently  only  20  of  the  58  local  districts  have 
computer  systems,  none  of  which  are  compatible.   The  remaining  districts 
maintain  manual  systems. 

With  the  advent  of  Quality  Control,  the  State  Department  of  Social  Services 
created  the  Office  of  Audit  and  Quality  Control  with  sole  responsibility  for 
auditing  all  programs,  local  district  management,  and  the  conduct  of  Quality 
Control  programs. 

In  the  interest  of  economy,  three  Regional  Audit  Offices  were  established. 
The  Metropolitan  Office  is  responsible  for  reviews  in  New  York  City  and 
the  three  counties  surrounding  the  City.   The  Eastern  Regional  Office  in 
Rochester  is  responsible  for  the  remaining  25  counties.   Originally,  each 
regional  office  assigned  separate  staff  to  each  federal  program.   Staff  is 
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rotated  periodically  within  each  region  in  order  that  they  may  become 
familiar  with  the  requirements  of  all  programs. 

When  the  Department  of  Agriculture  revised  their  sampling  requirements 
to  include  all  Food  Stamp  recipients,  it  seemed  logical  to  select  a 
proportion  of  AFDC  to  be  included  in  both  samples.   This  thinking  was 
also  supported  by  the  fact  that  at  that  time  New  York  State  government 
was  struggling  with  a  hiring  freeze  and  our  QC  staff  was  being  depleted 
by  promotions  and  transfers  to  other  program  divisions  within  the 
Department.   The  integration  could  reduce  the  overall  staff  involved  in 
Food  Stamp  reviews. 

Considering  the  problems  and  circumstances  outlined  above,  we  developed 
a  sampling  methodology  as  follows: 

1 .  Estimate  the  monthly  average  number  of  cases  in  the  F/S  universe 
for  a  six-month  period, 

2.  Estimate  the  monthly  average  number  of  cases  in  the  AFDC  universe 
that  utilize  Food  Stamps. 

3.  Predetermine  the  number  of  positive  F/S  and  AFDC  cases  that  must  be 
selected  for  review  each  month  in  order  to  satisfy  the  completion 
requirements  of  both  programs  (215  'AFDC,  240  F/S). 

4.  Determine  proportional  sample  sizes  for  New  York  City,  Metropolitan 
Counties  (Metro  Region) ,  and  Eastern  and  Western  Regions  based  upon 
the  total  AFDC  and  F/S  population  in  each  region. 

5.  Select  one  three-digit  number  from  a  table  of  random  numbers,  monthly. 
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6.  Transmit  the  case  number  ending  on  form  DSS-1727  to  the  58  local 
districts  requesting  them  to  report  to  Central  Office  by  the  10th 
of  each  month  all  AFDC  and  F/S  cases  active  as  of  the  1st  of  the 
review  month  that  have  the  designated  case  number  ending. 

7.  Select  the  predetermined  AFDC  sample  for  each  region  by  systematically 
deleting  excess  cases  from  the  lists  submitted  by  the  local  districts. 
This  produces  four  stratified  proportional  samples  for  the  AFDC 
program. 

8.  Identify  all  AFDC  cases  selected  in  step  7.  that  also  appear  on  the 
F/S  case  lists.  Select  a  proportional  sample  of  these  cases  in  each 
strata  using  the  systematic  deletion  process. 

9.  Systematically  select  the  ramaining  number  of  F/S  cases  needed  to 
complete  the  predetermined  monthly  sample  requirements  from  all 
other  cases  (HR,  SSI,  and  NPA)  reported  by  the  local  districts. 

Note:   The  systematic  selection  method,  rather  than  the  fixed  inter- 
val, must  be  utilized  to  assure  that  each  regional  office  has 
a  constant  workload  each  month.   Staff  limitations  and  an 
austerity  budget  do  not  permit  shifting  staff  assignments  in 
the  instances  of  variations  in  the  monthly  workload.   A  study 
of  a  prior  period  indicated  that  at  least  four  reviewers 
would  have  to  be  shifted  to  QC  reviews  sporadically  within  a 
region. 

The  sample  design  facilitates  the  conduct  of  the  reviews  only.   Data 
analysis  and  reporting  requirements  are  accomplished  separately  for  each 
program . 
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Integrity  of  the  State  Case  Review  Process 

1.  All  case  listings  are  received  in  Central  Office  by  the  10th  of  each 
month  and  are  assigned  to  Regional  Office  staff  by  the  15th  of  the 
month.   This  integrated  sampling  process  will  not  effect  our  present 
workflow. 

2.  Originally,  Quality  Control  reviews  were  conducted  by  separate  units 
within  each  regional  office.   Due  to  budget  cuts  the  Eastern  and 
Western  Regional  Offices  began  assigning  cases  involved  in  the  three 
programs  by  geographic  area  to  eliminate  the  need  for  more  than  one 
reviewer  traveling  to  a  remote  area.   The  Metropolitan  Office,  due 

to  the  size  of  the  workload  and  limited  travel,  continued  to  maintain 
separate  review  units  for  each  of  the  three  programs.   Integration  in 
all  regions  permits  approximately  44  percent  of  the  required  F/S  cases 
to  be  reviewed  simultaneously  with  the  cooperative  AFDC  case.   Experience 
has  proven  that  we  can  complete  the  required  reviews  with  a  30  percent 
reduction  in  the  F/S  review  staff.   AFDC  reviewers  absorbed  the  additional 
workload. 

During  the  review  process  reviewers  have  been  instructed  to  complete 
both  the  OQC  341A  and  the  corresponding  FNS  245  with  the  exception  of 
items  that  are  identical  to  both  forms.   This  documentation  is  recorded 
only  on  the  OQC  341A  and  cross-referenced  on  the  FNS  245.   Separate 
review  schedules  are  completed  for  each  category. 

As  with  any  new  methodology  we  encountered  problems  related  to  training 
review  staff.   In  order  to  implement  the  integrated  reviews  it  was 
necessary  to  train  AFDC  review  staff  in  the  requirements  of  the  Food  Stamp 
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Program.   As  we  all  know,  it  usually  takes  six  months  before  reviewers 
become  proficient.   In  order  to  overcome  the  time  element,  we  relied 
upon  first-line  supervisors  who  were  experienced  with  the  Food  Stamp 
regulations  to  review  all  cases  and  conclusions  before  the  cases  were 
considered  complete. 

Our  first  administrative  review  by  USDA  after  the  implementation  of  the 
system  resulted  in  severe  criticism.   They  were  concerned  that  all  cases 
in  the  universe  did  not  have  an  equal  opportunity  to  be  selected.   Changes 
in  caseloads  would  necessitate  weighing  of  the  QC  results  by  region  and 
finally  the  integrity  of  the  case  review  was  compromised  because  AFDC 
reviewers  would  tend  to  minimize  the  importance  of  the  elements  of  eligi- 
bility unique  to  the  Food  Stamp  Program. 

After  considering  the  objections  outlined  above,  we  felt  justified  in 
responding  as  follows: 

1.  At  the  point  the  cases  are  assigned  to  a  strata,  all  cases  in 
each  strata  had  an  equal  opportunity  to  be  selected. 

2.  After  the  influx  of  SSI  recipients,  who  were  formerly  cashed-out, 
the  statewide  caseload  has  not  varied  by  more  than  two  percent. 
In  the  event  of  any  major  change,  we  would  naturally  re-evaluate 
our  proportional  samples. 

3.  Regarding  the  integrity  of  the  sample,  we  spent  considerable  time 
with  the  staff  emphasizing  the  importance  of  thorough  documentation 
of  all  elements  included  in  the  FNS  245  worksheet.   This  weakness 
in  the  review  process  is  slowly  being  overcome  and  hopefully  with 
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the  new  Food  Stamp  regulations  the  problem  will  be  minimized.   In 
support  of  our  review  staff,  New  York  State  has  an  aggregate  error 
rate  of  over  50  percent.   It  wouldn't  appear  that  the  reviews  have 
been  inadequate . 

Presented  by 


Charles  J.  Kavanaugh ,  Asst.  Director 

Bureau  of  Audit  Operations 

NYS  Department  of  Social  Services 
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Section  IV  -  Work  Shops 

Two  separate  workshops  were  held  in  order  to  provide 
statistical  and  program  staff  the  opportunity  to 
discuss  issues  of  specific  concern  to  them.   The 
statistical  workshop  concentrated  on  sampling  issues 
with  special  emphasis  on  various  integrated  sample 
designs.   The  case  review  workshop  focused  on  the 
impact  of  integration  on  the  case  review  process  it- 
self and  other  management  related  areas. 
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Work  Group  Section  -  #1  Sampling  Techniques 

Federal  and  State  statisticians  engaged  in  a  detailed  discussion  of  the 
various  problems  and  possible  solutions  to  designing  an  integrated  sample. 
Although  the  concerns  differed  between  State  and  Federal  statisticians, 
they  agreed  on  the  importance  of  QC  sample  integrity  and  savings  in  QC 
costs.   The  discussion  specifically  considered  the  various  integrated 
sampling  designs  that  were  being  used.   While  some  States  welcomed  the 
opportunity  to  integrate  QC  systems  with  complex  stratified  sampling 
designs,  other  States  prefer  self-weighting  designs. 

The  session  considered  specific  sampling  procedures  for  integrating 
sample  selection,  with  examples  from  West  Virginia  and  South  Dakota. 
West  Virginia  creates  strata  for  sampling  cases  in  the  AFDC ,  Medicaid, 
and  FS  frames  using  a  different  estimated  sample  interval  in  each  frame. 
Thus,  a  weighted  sample  is  produced  without  knowing  the  true  universe 
population  sizes  of  strata.   In  contrast.  South  Dakota  produces  a  self- 
weighting  sample  by  randomly  selecting  the  last  four  digits  of  social 
security  numbers  each  month.   Because  both  AFDC  and  FS  cases  have  social 
security  numbers,  representative  samples  occur  from  the  same  set  of 
random  numbers.   These  examples  indicate  that  integrated  sampling  designs 
•can  have  different  objectives. 

When  alternative  sample  designs  were  discussed,  a  concensus  did  not  emerge 
on  the  objective  for  sampling.   Statisticians  questioned  whether  sampling 
plans  would  continue  to  be  based  on  fixed  sample  size  requirements.   When 
each  Federal  QC  program  requires  a  minimum  sample  size  with  systematic 
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sampling,  the  number  of  State  reviews  and/or  costs  of  reviews  may  be  higher 
than  in  an  integrated  sample  design  based  on  other  requirements.   Does  one 
minimize  the  number  of  case  reviews,  costs,  or  sample  variability? 
Different  sample  designs  can  accomplish  different  objectives.   Some  Federal 
statisticians  agreed  that  their  future  requirements  should  be  more  flexible 
but  must  maintain  sample  integrity.   The  common  sampling  goals  appropriate 
for  the  three  QC  programs  have  not  yet  been  defined.   The  statisticians 
also  discussed  at  length  whether  alternatives  to  self-weighting  samples  are 
beneficial.   Some  said  that  they  preferred  self-weighting  samples  because 
of  easier  calculation  of  error  rates  for  reports.   In  addition,  complex 
statistical  analyses  such  as  multivariate  statistics  for  error  prone  profiles 
can  be  used.   One  possible  liability  to  self-weighting  samples,  however,  is 
fluctuation  of  sample  size  by  month  or  by  geographic  area.   If  such  fluctu- 
ations are  controlled,  costs  are  often  increased.   Although  more  costly,  non 
self-weighting  samples  can  be  made  self-weighting  by  discarding  completed  QC 
reviews.   Weighted  stratification  was  observed  to  require  more  complex 
statistical  calculations  for  reports  but  is  generally  less  expensive  than  a 
self-weighted  sample.   These  and  other  alternatives  to  self-weighting 
samples  appear  available  to  States  if  they  choose  providing  that 
sample  integrity  is  maintained. 

The   possibility  of  a  model  sampling  plan  which  States  could  readily 
adopt  was  suggested.   The  approach  was  not  preferred,  however,  because 
it  would  not  provide  flexibility  in  the  sampling  designs.   It  was  also 
questioned  whether  Central  office  approval  of  State  integrated  sample 
plans  was  necessary  in  that  it  was  a  time  delaying  factor.   Central 
office  approval  was  justified,  however,  because  of  the  lack  of  a  uniform 
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distribution  of  statistical  personnel  at  the  various  Regional  levels. 
Central  Office  approval  ensures  uniformity  and  safeguards  the  State 
against  implementing  an  unapprovable  plan. 

The  statisticians  suggested  several  solutions  for  problems  in  integrated 
sampling.   A  common  problem  to  many  States  is  the  mixture  of  household 
heads  between  programs.   Many  States  are  unable  to  identify  when  a  house- 
hold has  an  AFDC  assistance  unit  head  who  differs  from  the  Food  Stamp 
household  head.   Because  such  households  can  be  sampled  from  two  frames, 
the  probability  of  their  selection  is  higher  than  it  should  be.   The 
solution  suggested  was  reviewing  a  mixed  household  for  only  one  program 
whether  it  be  AFDC  or  Food  Stamp. 

Because  some  States  are  unable  to  determine  the  statewide  sizes  of  strata 
in  integrated  sampling,  they  have  not  been  able  to  use  alternatives  to 
self-weighting  samples.   A  solution  proposed  was  estimation  of  strata 
weights  from  sampled  data;  this  method  is  currently  being  tested  in  West 
Virginia.   Because  the  calculation  of  sampling  variance  is  more  complex, 
the  weighting  procedure  may  affect  the  regression  adjustment  method  used 
in  determining  official  error  rates  by  SSA. 

Some  State  statisticians  voiced  concern  that  State  resources  could  not 
create  reports  if  all  QC  systems  adopt  the  same  time  deadlines.   The 
resource  demands  from  MQC  were  also  of  concern. 

In  summary,  statisticians  agreed  on  the  importance  of  QC  sample  integrity 
and  savings  in  QC  costs.   Based  upon  the  discussion,  one  can  conclude  that 
alternative  sampling  procedures  exist  for  integrated  sampling.   States  can 
elect  procedures  for  self-weighting  samples  or  more  complex  procedures 
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requiring  weighting.   The  appropriate  approach  to  integrated  sampling 
will  depend  upon  State  circumstances  and  Federal  requirements  for  QC 
sampling. 
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Nummary  of  Work  Group  Session  -  #2 
Case  Review  Process  and  Operations 

Federal  and  State  QC  program  staff  engaged  in  discussions  centered  around 
the  following  issues  felt  to  have  direct  relevance  to  the  case  review 
process  and  operations. 

1 .   Can  integrated  case  reviews  conducted  by  generic  reviewers  be  of  the 
same  quality  as  separate  reviews  performed  by  specialized  reviewers? 

One  State  with  an  integrated  QC  system  felt  that  quality  had  initially 
suffered  but  that  it  had  improved  to  its  previous  level  over  several 
review  periods.   Two  States  who  currently  have  generic  reviewers  (but 
not  integrated  sample  designs)  expressed  concern  over  the  quality  of 
reviews  in  their  States  and  are  exploring  mechanisms  to  make  improve- 
ments (it  is  interesting  to  note  that  in  both  of  these  States,  the 
concern  resulted  from  local  agency  challenges  to  QC  findings).   Several 
States  were  concerned  that  their  own  State  policies  were  so  complex 
that  it  would  be  difficult  for  a  reviewer  to  learn  all  three  Programs 
and  still  maintain  a  high  standard  of  quality.   One  State,  for  example, 
who  has  integrated  FS  and  AFDC  Quality  Control,  felt  that  their  State's 
medical  assistance  policy  was  too  complicated  to  be  integrated  with  FS 
and  AFDC.   Another  State  who  has  integrated  all  three  programs  felt 
that  their  simplified  and  integrated  State  policy  was  an  asset  to 
sample  integration.   This  State  felt  that  learning  and  applying  the 
three  different  QC  policies  was  the  reviewer's  most  difficult  task. 
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For  the  most  part,  however,  States  felt  that  a  high  level  of  quality 
can  be  maintained  under  an  integrated  QC  system  and  that  integration 
enhances  reviewer  interest  and  upgrades  the  job  function.   In  fact. 
States  with  experience  in  integration  generally  felt  that  after  some 
initial  resistence,  reviewers  came  to  prefer  the  integrated  approach 
as  enhancing  their  jobs.   States  were  quick  to  point  out,  however, 
that  an  emphasis  must  be  maintained  on  training  and  reviewer  standards 
in  order  to  insure  a  quality  product. 

2-   What  mechanisms  are  used  or  can  be  used  to  measure  the  quality  of 
integrated  case  reviews? 

As  with  most  States,  States  that  have  integrated  their  QC  systems  rely 
heavily  on  Federal  re-review  findings  as  an  indication  of  the  quality 
of  their  reviews.   These  States  felt  that  the  quality  of  reviews  have 
remained  high  under  integration  and  cited  the  lack  of  an  increase  in 
Federal  differences  as  evidence  supporting  their  contention.   On  the 
other  hand,  the  two  States  with  generic  reviewers  who  previously 
expressed  concern  over  the  quality  of  reviews  cited  challenges  from 
the  local  agencies  as  the  mechanism  which  brought  their  reviews  into 
question.   Most  States,  however,  felt  that  the  Federal  re-review  was 
an  adequate  mechanism  to  measure  the  quality  of  State  reviews. 

3.   What  implications  does  integration  have  for  the  training  function? 

It  was  agreed  that  training  became  more  important  under  integration 
and  that  more  emphasis  must  be  placed  on  initial  as  well  as  ongoing 
training.   States  generally  agreed  that  adequate  resources  did  not 
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exist  outside  the  QC  Unit  and  that  except  for  program  input,  training 
must  be  performed  in-house.   However,  States  did  not  see  the  increased 
training  demands  placed  on  the  QC  Unit  as  a  negative  aspect  of  inte- 
gration. 

4 ,  Does  relating  to  three  separate  recipient  populations  cause  inherent 
problems  for  the  QC  reviewer? 

No.  States  felt  strongly  that  recipients  in  each  program  represented 
a  wide  range  of  characteristics  and  that  reviewers  must  be  skilled  in 
dealing  with  all  types  of  people  regardless  of  the  program  under  review. 

5.  What  are  the  advantages  and  disadvantages  of  a  consolidated  worksheet? 

States  with  integrated  samples  were  quick  to  point  out  that  recording 
the  same  case  information  on  2  or  3  separate  worksheets  was  a  duplication 
of  effort  and  a  task  resented  by  the  reviewers.   As  a  result.  States 
felt  that  some  type  of  integrated  QC  worksheet  was  necessary.   In  fact, 
several  States  stated  that  they  would  not  consider  integration  unless 
an  integrated  worksheet  was  available. 

Clearly,  the  primary  advantage  of  such  a  worksheet  is  the  minimization 
of  case  recording;  which  could  result  in  substantial  savings  in  time 
and  effort.   To  do  so  the  worksheet  must  be  flexible  enough  to  accommo- 
date the  various  program  requirements.   At  the  same  time,  however, 
concern  was  expressed  that  the  worksheet  provide  enough  structure  to 
guide  the  reviewer  through  the  review  and  insure  that  each  program 
element  is  thoroughly  covered.   It  was  also  felt  that  in  order  to 
facilitate  different  case  review  completion  schedules,  the  worksheet 
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must  provide  for  reaching  clearly  independent  QC  review  findings  at 
any  time  during  the  case  review.   This  would  become  particularly 
important  in  the  event  that  AFDC-QC  findings  were  required  to  be 
completed  within  60  days  after  the  sample  month.   A  major  disadvantage 
of  an  integrated  QC  worksheet  would  be  its  lack  of  flexibility  in 
reflecting  QC  program  changes.   If  one  program  makes  a  change  in  its 
QC  system  which  affects  the  worksheet,  it  may  be  difficult  to  revise 
the  worksheet  without  adversely  effecting  the  other  programs.   This 
process,  in  that  it  would  involve  2  or  more  Agencies,  may  be  compli- 
cated and  time  consuming. 

Although  several  States  are  experimenting  with  integrated  worksheets, 
none  have  been  field  tested  to  date.   Even  though  there  are  dis- 
advantages to  an  integrated  worksheet,  it  is  believed  that  it  is  a 
logical  companion  to  an  integrated  sample  design.   To  this  end,  the 
Department  of  HEW  and  the  Department  of  Agriculture  have  agreed  to 
jointly  explore  the  possibility  of  developing  an  integrated  QC  work- 
sheet for  State  use. 

6.   What  are  the  implications  of  integrated  sample  designs  on  the  Corrective 
Action  Process? 

States  were  in  agreement  that  Sample  integration  would  have  a  positive 
effect  on  the  CA  Process.   It  was  felt  that  having  data  available  on 
multi-benefit  cases  would  be  an  extra  degree  of  data  sophistication 
which  could  impact  on  such  areas  as  error  profile  systems  and  cost 
benefit  analyses. 
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7.   What  impact  does  the  shift  in  the  AFDC-QC  review  cycle  have  on  sample 
integration? 

The  shift  in  the  AFDC-QC  review  cycle  from  January-June  to  April- 
September  makes  it  compatable  with  MEQC  but  incompatable  with  FS-QC . 
The  general  concensus  was  that  all  three  programs  must  be  on  the  same 
review  cycle  in  order  to  coordinate  sample  selections,  changes  in  the 
various  QC  programs   and  data  processing.   States  expressed  concern, 
however,  that  care  must  be  taken  at  the  Federal  level  to  insure  that 
case  completion  and  reporting  requirements  are  staggered  so  that 
unmanageable  workloads  do  not  develop. 
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Frankfort,  Kentucky  40601 
502-564-3546 
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AFDC-Technical  Services  Branch 

Office  of  Quality  Assurance,  SSA 

Room  1416  -  Mary  E.  Switzer  Building 

330  C  Street  S.W. 

Washington,  D.C.   20201 
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1900  Washington  Street  E 

Charleston,  West  Virginia  25305 

304-348-2400 
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Program  Analyst 
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Room  1117-C  -  Mary  E.  Switzer  Building 

330  C  Street,  S.W. 

Washington,  D.C.   20201 

202-472-7761 
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Quality  Control  Division 

Arkansas  Social  Services 
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Little  Rock,  Arkansas   72203 

501-371-1900 

Mr.  Charles  Trout 
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200  Independence  Avenue,  S.W. 

Washington,  D.C.  20201 

202-245-7515 
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Department  of  Human  Resources 

Quality  Control  Division 

110  Metro  Plaza 

Baltimore,  Maryland   21215 
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Quality  Control 
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State  Office  Building 
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Program  Evaluation  and  Review 
301  Centennial  Mall  South 
P.O.  Box  95026 
Lincoln,  Nebraska   68501 
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ARC-FA-Social  Security  Administration 

P.  0.  Box  725 

Denver,  Colorado   80201 
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Wilbur  A.  Weder,  Statistician 

Office  of  Family  Assistance,  SSA 

300  S  Wacker  Drive 

30th  Floor 

Chicago,  Illinois   60606 
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Ms.  Nancy  W.  White 
Income  Maintenance 
Quality  Control 
222  State  Street 
Augusta,  Maine   04333 
207-289-2826 
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Chief,  Quality  Control 

Department  of  Social  Services 
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Middletown,  Connecticut   06457 
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Office  of  Audit  and  Quality  Control 
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Fondren  Station 

Jackson,  Mississippi   39216 
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Division  of  Quality  Control 
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